[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:09:51.780972 27901 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.63.126:44211
I20260709 13:09:51.782444 27901 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:09:51.783416 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:51.800851 27908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:51.804451 27907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:51.800906 27910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:53.011710 27909 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 13:09:53.011838 27901 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 13:09:53.016536 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:53.016783 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:53.017042 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602593017016 us; error 0 us; skew 500 ppm
I20260709 13:09:53.017899 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:53.024464 27901 webserver.cc:533] Webserver started at http://127.27.63.126:34293/ using document root <none> and password file <none>
I20260709 13:09:53.025498 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:53.025731 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:53.026213 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:53.032164 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/master-0-root/instance:
uuid: "28d419f2cdcb4937aa84c5b9d69f1a95"
format_stamp: "Formatted at 2026-07-09 13:09:53 on dist-test-slave-sjgx"
I20260709 13:09:53.040325 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:09:53.045655 27916 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.046770 27901 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:09:53.047154 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/master-0-root
uuid: "28d419f2cdcb4937aa84c5b9d69f1a95"
format_stamp: "Formatted at 2026-07-09 13:09:53 on dist-test-slave-sjgx"
I20260709 13:09:53.047494 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:53.103101 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:53.104758 27901 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:09:53.105247 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:53.176730 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.126:44211
I20260709 13:09:53.176810 27967 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.126:44211 every 8 connection(s)
I20260709 13:09:53.184746 27968 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:53.206460 27968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95: Bootstrap starting.
I20260709 13:09:53.212373 27968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:53.214150 27968 log.cc:826] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95: Log is configured to *not* fsync() on all Append() calls
I20260709 13:09:53.218901 27968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95: No bootstrap required, opened a new log
I20260709 13:09:53.240535 27968 raft_consensus.cc:359] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d419f2cdcb4937aa84c5b9d69f1a95" member_type: VOTER }
I20260709 13:09:53.241349 27968 raft_consensus.cc:385] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:53.241674 27968 raft_consensus.cc:740] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d419f2cdcb4937aa84c5b9d69f1a95, State: Initialized, Role: FOLLOWER
I20260709 13:09:53.242849 27968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d419f2cdcb4937aa84c5b9d69f1a95" member_type: VOTER }
I20260709 13:09:53.243317 27968 raft_consensus.cc:399] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:53.243556 27968 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:53.243839 27968 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:53.248019 27968 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d419f2cdcb4937aa84c5b9d69f1a95" member_type: VOTER }
I20260709 13:09:53.248783 27968 leader_election.cc:304] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28d419f2cdcb4937aa84c5b9d69f1a95; no voters: 
I20260709 13:09:53.250439 27968 leader_election.cc:290] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:53.250793 27971 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:53.253042 27971 raft_consensus.cc:697] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 1 LEADER]: Becoming Leader. State: Replica: 28d419f2cdcb4937aa84c5b9d69f1a95, State: Running, Role: LEADER
I20260709 13:09:53.254019 27971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d419f2cdcb4937aa84c5b9d69f1a95" member_type: VOTER }
I20260709 13:09:53.255069 27968 sys_catalog.cc:565] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:53.265512 27972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28d419f2cdcb4937aa84c5b9d69f1a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d419f2cdcb4937aa84c5b9d69f1a95" member_type: VOTER } }
I20260709 13:09:53.266279 27972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:53.266911 27973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d419f2cdcb4937aa84c5b9d69f1a95. Latest consensus state: current_term: 1 leader_uuid: "28d419f2cdcb4937aa84c5b9d69f1a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d419f2cdcb4937aa84c5b9d69f1a95" member_type: VOTER } }
I20260709 13:09:53.267709 27973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:53.270007 27980 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:53.283666 27980 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:53.286164 27901 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:53.300371 27980 catalog_manager.cc:1383] Generated new cluster ID: 4459f5ab66054e4e8ad0299e93b05787
I20260709 13:09:53.300668 27980 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:53.317344 27980 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:53.318951 27980 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:53.332625 27980 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95: Generated new TSK 0
I20260709 13:09:53.333560 27980 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:53.353366 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:53.363306 27990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:53.364962 27991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:53.368732 27901 server_base.cc:1061] running on GCE node
W20260709 13:09:53.369779 27993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:53.370898 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:53.371106 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:53.371328 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602593371289 us; error 0 us; skew 500 ppm
I20260709 13:09:53.371937 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:53.376401 27901 webserver.cc:533] Webserver started at http://127.27.63.65:42247/ using document root <none> and password file <none>
I20260709 13:09:53.376946 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:53.377162 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:53.377458 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:53.378578 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-0-root/instance:
uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
format_stamp: "Formatted at 2026-07-09 13:09:53 on dist-test-slave-sjgx"
I20260709 13:09:53.384593 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.002s
I20260709 13:09:53.389297 27998 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.390137 27901 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.003s
I20260709 13:09:53.390440 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
format_stamp: "Formatted at 2026-07-09 13:09:53 on dist-test-slave-sjgx"
I20260709 13:09:53.390745 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:53.427801 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:53.429203 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:53.432085 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:53.436721 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:53.436949 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.437290 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:53.437443 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.545552 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:37339
I20260709 13:09:53.545593 28060 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:37339 every 8 connection(s)
I20260709 13:09:53.552026 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:53.561839 28065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:53.566721 28066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:53.573544 28068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:53.578192 27901 server_base.cc:1061] running on GCE node
I20260709 13:09:53.579442 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:53.579751 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:53.580026 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602593579997 us; error 0 us; skew 500 ppm
I20260709 13:09:53.580932 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:53.584645 28061 heartbeater.cc:344] Connected to a master server at 127.27.63.126:44211
I20260709 13:09:53.585089 27901 webserver.cc:533] Webserver started at http://127.27.63.66:39351/ using document root <none> and password file <none>
I20260709 13:09:53.585275 28061 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:53.585805 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:53.586367 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:53.586586 28061 heartbeater.cc:507] Master 127.27.63.126:44211 requested a full tablet report, sending...
I20260709 13:09:53.586715 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:53.587920 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-1-root/instance:
uuid: "15aee576f44a4602894736f69680811b"
format_stamp: "Formatted at 2026-07-09 13:09:53 on dist-test-slave-sjgx"
I20260709 13:09:53.590600 27933 ts_manager.cc:194] Registered new tserver with Master: e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339)
I20260709 13:09:53.593737 27933 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34422
I20260709 13:09:53.595192 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:53.600034 28073 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.600970 27901 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:09:53.601320 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-1-root
uuid: "15aee576f44a4602894736f69680811b"
format_stamp: "Formatted at 2026-07-09 13:09:53 on dist-test-slave-sjgx"
I20260709 13:09:53.601619 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:53.615484 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:53.616966 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:53.619731 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:53.622751 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:53.622996 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.623250 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:53.623430 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.729331 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.66:40925
I20260709 13:09:53.729498 28135 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.66:40925 every 8 connection(s)
I20260709 13:09:53.741250 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:53.751062 28139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:53.753429 28136 heartbeater.cc:344] Connected to a master server at 127.27.63.126:44211
I20260709 13:09:53.753849 28136 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:53.754815 28136 heartbeater.cc:507] Master 127.27.63.126:44211 requested a full tablet report, sending...
I20260709 13:09:53.757362 27901 server_base.cc:1061] running on GCE node
I20260709 13:09:53.757440 27933 ts_manager.cc:194] Registered new tserver with Master: 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
W20260709 13:09:53.757710 28140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:53.758956 28142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:53.759502 27933 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34430
I20260709 13:09:53.760092 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:53.760331 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:53.760524 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602593760505 us; error 0 us; skew 500 ppm
I20260709 13:09:53.761178 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:53.764027 27901 webserver.cc:533] Webserver started at http://127.27.63.67:42045/ using document root <none> and password file <none>
I20260709 13:09:53.764564 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:53.764809 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:53.765182 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:53.766286 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-2-root/instance:
uuid: "20ca690f4dc24bc2a051124def8a124d"
format_stamp: "Formatted at 2026-07-09 13:09:53 on dist-test-slave-sjgx"
I20260709 13:09:53.771894 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:09:53.775609 28147 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.776580 27901 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:09:53.776921 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-2-root
uuid: "20ca690f4dc24bc2a051124def8a124d"
format_stamp: "Formatted at 2026-07-09 13:09:53 on dist-test-slave-sjgx"
I20260709 13:09:53.777268 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602591602838-27901-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:53.794379 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:53.795758 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:53.797453 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:53.800678 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:53.800961 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.801286 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:53.801496 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:53.895460 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.67:41679
I20260709 13:09:53.895592 28209 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.67:41679 every 8 connection(s)
I20260709 13:09:53.914075 28210 heartbeater.cc:344] Connected to a master server at 127.27.63.126:44211
I20260709 13:09:53.914517 28210 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:53.915345 28210 heartbeater.cc:507] Master 127.27.63.126:44211 requested a full tablet report, sending...
I20260709 13:09:53.917371 27933 ts_manager.cc:194] Registered new tserver with Master: 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:53.917788 27901 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013787885s
I20260709 13:09:53.918874 27933 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34442
I20260709 13:09:53.973847 27933 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34448:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:53.977556 27933 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:54.072573 28026 tablet_service.cc:1511] Processing CreateTablet for tablet be4ae96b1b064518844fc56e9296677d (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:54.072794 28025 tablet_service.cc:1511] Processing CreateTablet for tablet 5db7d5960e184238a5cecc537f2aedd8 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:54.075052 28024 tablet_service.cc:1511] Processing CreateTablet for tablet 787dd4bb42b548c39e14ada836c30081 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:54.075310 28022 tablet_service.cc:1511] Processing CreateTablet for tablet 6faee2f44c7e44db9f459461b9279789 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:54.077384 28026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4ae96b1b064518844fc56e9296677d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.077105 28023 tablet_service.cc:1511] Processing CreateTablet for tablet 1e5120e7f37845eb977a87e7eca3b017 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:54.078588 28022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6faee2f44c7e44db9f459461b9279789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.079241 28025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db7d5960e184238a5cecc537f2aedd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.080178 28023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e5120e7f37845eb977a87e7eca3b017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.084492 28024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 787dd4bb42b548c39e14ada836c30081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.093253 28172 tablet_service.cc:1511] Processing CreateTablet for tablet 1e5120e7f37845eb977a87e7eca3b017 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:54.102857 28175 tablet_service.cc:1511] Processing CreateTablet for tablet be4ae96b1b064518844fc56e9296677d (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:54.092888 28173 tablet_service.cc:1511] Processing CreateTablet for tablet 787dd4bb42b548c39e14ada836c30081 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:54.113166 28171 tablet_service.cc:1511] Processing CreateTablet for tablet 6faee2f44c7e44db9f459461b9279789 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:54.111682 28172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e5120e7f37845eb977a87e7eca3b017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.116384 28021 tablet_service.cc:1511] Processing CreateTablet for tablet 70e6679c989543a4a60743fbf5628180 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:54.104406 28174 tablet_service.cc:1511] Processing CreateTablet for tablet 5db7d5960e184238a5cecc537f2aedd8 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:54.120126 28171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6faee2f44c7e44db9f459461b9279789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.125319 28021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e6679c989543a4a60743fbf5628180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.127038 28175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4ae96b1b064518844fc56e9296677d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.129181 28174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db7d5960e184238a5cecc537f2aedd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.133603 28173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 787dd4bb42b548c39e14ada836c30081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.143059 28170 tablet_service.cc:1511] Processing CreateTablet for tablet 70e6679c989543a4a60743fbf5628180 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:54.147142 28170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e6679c989543a4a60743fbf5628180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.167604 28020 tablet_service.cc:1511] Processing CreateTablet for tablet 0c1b8e756bf54b05b16b4d821d855df8 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:54.173367 28020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c1b8e756bf54b05b16b4d821d855df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.171916 28169 tablet_service.cc:1511] Processing CreateTablet for tablet 0c1b8e756bf54b05b16b4d821d855df8 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:54.184754 28100 tablet_service.cc:1511] Processing CreateTablet for tablet 5db7d5960e184238a5cecc537f2aedd8 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:54.185879 28099 tablet_service.cc:1511] Processing CreateTablet for tablet 787dd4bb42b548c39e14ada836c30081 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:54.184806 28101 tablet_service.cc:1511] Processing CreateTablet for tablet be4ae96b1b064518844fc56e9296677d (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:54.199805 28169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c1b8e756bf54b05b16b4d821d855df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.205271 28095 tablet_service.cc:1511] Processing CreateTablet for tablet 0c1b8e756bf54b05b16b4d821d855df8 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:54.205735 28100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db7d5960e184238a5cecc537f2aedd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.219251 28099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 787dd4bb42b548c39e14ada836c30081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.202706 28097 tablet_service.cc:1511] Processing CreateTablet for tablet 6faee2f44c7e44db9f459461b9279789 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:54.221043 28098 tablet_service.cc:1511] Processing CreateTablet for tablet 1e5120e7f37845eb977a87e7eca3b017 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:54.199098 28096 tablet_service.cc:1511] Processing CreateTablet for tablet 70e6679c989543a4a60743fbf5628180 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:54.229990 28101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4ae96b1b064518844fc56e9296677d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.224426 28019 tablet_service.cc:1511] Processing CreateTablet for tablet 92501d951008475fa44983bc16a8fb9e (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:54.237334 28168 tablet_service.cc:1511] Processing CreateTablet for tablet 92501d951008475fa44983bc16a8fb9e (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:54.237417 28094 tablet_service.cc:1511] Processing CreateTablet for tablet 92501d951008475fa44983bc16a8fb9e (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:54.240339 28168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92501d951008475fa44983bc16a8fb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.239830 28019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92501d951008475fa44983bc16a8fb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.250140 28098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e5120e7f37845eb977a87e7eca3b017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.255299 28096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e6679c989543a4a60743fbf5628180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.256268 28094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92501d951008475fa44983bc16a8fb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.259958 28095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c1b8e756bf54b05b16b4d821d855df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.264215 28097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6faee2f44c7e44db9f459461b9279789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.264251 28227 tablet_bootstrap.cc:492] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:54.281045 28228 tablet_bootstrap.cc:492] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
W20260709 13:09:54.284832 27981 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:54.282279 28022 tablet_service.cc:1511] Processing CreateTablet for tablet ce59f69f404a4584bce3d7f4c5cd8476 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:54.297843 28093 tablet_service.cc:1511] Processing CreateTablet for tablet ce59f69f404a4584bce3d7f4c5cd8476 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:54.298403 28022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce59f69f404a4584bce3d7f4c5cd8476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.301174 28093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce59f69f404a4584bce3d7f4c5cd8476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.334812 28174 tablet_service.cc:1511] Processing CreateTablet for tablet ce59f69f404a4584bce3d7f4c5cd8476 (DEFAULT_TABLE table=test-workload [id=ad18a396fecd4b11bfcdd777b2eb8680]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:54.336150 28174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce59f69f404a4584bce3d7f4c5cd8476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.386886 28229 tablet_bootstrap.cc:492] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:54.395911 28227 tablet_bootstrap.cc:654] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.397810 28228 tablet_bootstrap.cc:654] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.442842 28229 tablet_bootstrap.cc:654] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.458441 28228 tablet_bootstrap.cc:492] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:54.458951 28228 ts_tablet_manager.cc:1403] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.197s	user 0.023s	sys 0.021s
I20260709 13:09:54.465076 28229 tablet_bootstrap.cc:492] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:54.464754 28228 raft_consensus.cc:359] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.465550 28229 ts_tablet_manager.cc:1403] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.080s	user 0.010s	sys 0.036s
I20260709 13:09:54.465701 28228 raft_consensus.cc:385] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.466125 28228 raft_consensus.cc:740] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.466558 28227 tablet_bootstrap.cc:492] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:54.466961 28227 ts_tablet_manager.cc:1403] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.213s	user 0.015s	sys 0.030s
I20260709 13:09:54.467062 28228 consensus_queue.cc:260] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.468313 28229 raft_consensus.cc:359] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.469015 28229 raft_consensus.cc:385] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.469307 28229 raft_consensus.cc:740] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.470036 28229 consensus_queue.cc:260] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.470736 28227 raft_consensus.cc:359] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.471676 28227 raft_consensus.cc:385] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.472060 28227 raft_consensus.cc:740] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.473137 28227 consensus_queue.cc:260] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.477294 28210 heartbeater.cc:499] Master 127.27.63.126:44211 was elected leader, sending a full tablet report...
I20260709 13:09:54.477689 28228 ts_tablet_manager.cc:1434] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.010s
I20260709 13:09:54.479203 28228 tablet_bootstrap.cc:492] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
I20260709 13:09:54.480801 28227 ts_tablet_manager.cc:1434] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.007s
I20260709 13:09:54.481518 28061 heartbeater.cc:499] Master 127.27.63.126:44211 was elected leader, sending a full tablet report...
I20260709 13:09:54.481752 28227 tablet_bootstrap.cc:492] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:54.489343 28227 tablet_bootstrap.cc:654] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.490375 28233 raft_consensus.cc:493] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.490875 28233 raft_consensus.cc:515] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.494027 28233 leader_election.cc:290] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:54.501338 28136 heartbeater.cc:499] Master 127.27.63.126:44211 was elected leader, sending a full tablet report...
I20260709 13:09:54.513026 28229 ts_tablet_manager.cc:1434] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.047s	user 0.013s	sys 0.025s
I20260709 13:09:54.514022 28229 tablet_bootstrap.cc:492] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:54.514792 28228 tablet_bootstrap.cc:654] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.521857 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:54.522615 28229 tablet_bootstrap.cc:654] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.523095 28036 raft_consensus.cc:2468] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:54.524943 28150 leader_election.cc:304] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:54.525810 28233 raft_consensus.cc:2804] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.526196 28233 raft_consensus.cc:493] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.526539 28233 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.535705 28233 raft_consensus.cc:515] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.540108 28227 tablet_bootstrap.cc:492] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:54.540599 28227 ts_tablet_manager.cc:1403] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.059s	user 0.030s	sys 0.007s
I20260709 13:09:54.547119 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:54.548094 28036 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.548743 28227 raft_consensus.cc:359] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.549489 28227 raft_consensus.cc:385] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.549798 28227 raft_consensus.cc:740] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.550923 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
W20260709 13:09:54.552881 28062 tablet.cc:2406] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:09:54.554817 28233 leader_election.cc:290] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
W20260709 13:09:54.555483 28150 leader_election.cc:343] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.556217 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:54.556525 28229 tablet_bootstrap.cc:492] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:54.556376 28227 consensus_queue.cc:260] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.556972 28229 ts_tablet_manager.cc:1403] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.013s
W20260709 13:09:54.557911 28150 leader_election.cc:343] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.558923 28227 ts_tablet_manager.cc:1434] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.004s
I20260709 13:09:54.559880 28227 tablet_bootstrap.cc:492] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:54.560215 28229 raft_consensus.cc:359] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.560007 28228 tablet_bootstrap.cc:492] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:54.560926 28229 raft_consensus.cc:385] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.563531 28229 raft_consensus.cc:740] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.564162 28228 ts_tablet_manager.cc:1403] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.085s	user 0.006s	sys 0.009s
I20260709 13:09:54.564495 28229 consensus_queue.cc:260] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.566913 28229 ts_tablet_manager.cc:1434] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:09:54.563113 28036 raft_consensus.cc:2468] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:54.567234 28227 tablet_bootstrap.cc:654] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.567806 28229 tablet_bootstrap.cc:492] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:54.568267 28150 leader_election.cc:304] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 15aee576f44a4602894736f69680811b
I20260709 13:09:54.569057 28233 raft_consensus.cc:2804] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.569753 28228 raft_consensus.cc:359] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.570972 28228 raft_consensus.cc:385] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.571341 28228 raft_consensus.cc:740] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.572288 28228 consensus_queue.cc:260] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.582464 28229 tablet_bootstrap.cc:654] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.590216 28227 tablet_bootstrap.cc:492] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:54.590737 28227 ts_tablet_manager.cc:1403] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.031s	user 0.002s	sys 0.012s
I20260709 13:09:54.591512 28228 ts_tablet_manager.cc:1434] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.008s
I20260709 13:09:54.592722 28228 tablet_bootstrap.cc:492] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
I20260709 13:09:54.593585 28227 raft_consensus.cc:359] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.594316 28227 raft_consensus.cc:385] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.594626 28227 raft_consensus.cc:740] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.595329 28227 consensus_queue.cc:260] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.597674 28233 raft_consensus.cc:697] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:54.598519 28233 consensus_queue.cc:237] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.599745 28228 tablet_bootstrap.cc:654] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.605466 28227 ts_tablet_manager.cc:1434] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 13:09:54.606472 28227 tablet_bootstrap.cc:492] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:54.619649 27932 catalog_manager.cc:5697] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 0 to 1, leader changed from <none> to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 1 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:54.626183 28227 tablet_bootstrap.cc:654] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.629974 28228 tablet_bootstrap.cc:492] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:54.631100 28228 ts_tablet_manager.cc:1403] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.008s
I20260709 13:09:54.632565 28229 tablet_bootstrap.cc:492] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:54.632997 28229 ts_tablet_manager.cc:1403] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.065s	user 0.015s	sys 0.009s
I20260709 13:09:54.634382 28228 raft_consensus.cc:359] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:54.635355 28228 raft_consensus.cc:385] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.635740 28228 raft_consensus.cc:740] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.635694 28229 raft_consensus.cc:359] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:54.636554 28229 raft_consensus.cc:385] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.636864 28229 raft_consensus.cc:740] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.637745 28229 consensus_queue.cc:260] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:54.640475 28229 ts_tablet_manager.cc:1434] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:54.641703 28229 tablet_bootstrap.cc:492] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:54.636850 28228 consensus_queue.cc:260] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:54.646036 28235 raft_consensus.cc:493] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.646524 28235 raft_consensus.cc:515] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.649514 28235 leader_election.cc:290] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:54.650063 28233 raft_consensus.cc:493] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.650552 28233 raft_consensus.cc:515] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.652797 28233 leader_election.cc:290] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:54.653874 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:54.654455 28228 ts_tablet_manager.cc:1434] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:09:54.654708 28036 raft_consensus.cc:2468] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:54.655592 28228 tablet_bootstrap.cc:492] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
I20260709 13:09:54.653874 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
W20260709 13:09:54.657477 28150 leader_election.cc:343] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.658213 28150 leader_election.cc:304] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 15aee576f44a4602894736f69680811b
I20260709 13:09:54.658797 28227 tablet_bootstrap.cc:492] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:54.659236 28227 ts_tablet_manager.cc:1403] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.004s
I20260709 13:09:54.659125 28233 raft_consensus.cc:2804] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.660220 28233 raft_consensus.cc:493] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.660562 28233 raft_consensus.cc:3060] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.665691 28229 tablet_bootstrap.cc:654] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.669894 28233 raft_consensus.cc:515] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.673732 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:54.674325 28036 raft_consensus.cc:3060] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.675858 28228 tablet_bootstrap.cc:654] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.677992 28227 raft_consensus.cc:359] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.678755 28227 raft_consensus.cc:385] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.679035 28227 raft_consensus.cc:740] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.679471 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b"
W20260709 13:09:54.681025 28150 leader_election.cc:343] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.679723 28227 consensus_queue.cc:260] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.689066 28036 raft_consensus.cc:2468] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:54.694854 28150 leader_election.cc:304] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 15aee576f44a4602894736f69680811b
I20260709 13:09:54.696074 28241 raft_consensus.cc:2804] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.696947 28241 raft_consensus.cc:697] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:54.697765 28241 consensus_queue.cc:237] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.686259 28233 leader_election.cc:290] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:54.707999 28227 ts_tablet_manager.cc:1434] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.001s
I20260709 13:09:54.709492 28227 tablet_bootstrap.cc:492] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:54.721908 28227 tablet_bootstrap.cc:654] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.723884 28234 raft_consensus.cc:493] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.724474 28234 raft_consensus.cc:515] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:54.725464 28228 tablet_bootstrap.cc:492] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:54.727672 28228 ts_tablet_manager.cc:1403] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.014s
I20260709 13:09:54.730132 28234 leader_election.cc:290] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:54.736704 28229 tablet_bootstrap.cc:492] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:54.737802 28229 ts_tablet_manager.cc:1403] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.096s	user 0.023s	sys 0.012s
I20260709 13:09:54.766485 28228 raft_consensus.cc:359] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.778087 28228 raft_consensus.cc:385] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.778568 28228 raft_consensus.cc:740] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.779958 28228 consensus_queue.cc:260] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.787736 28229 raft_consensus.cc:359] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.788518 28229 raft_consensus.cc:385] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.784245 28228 ts_tablet_manager.cc:1434] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.055s	user 0.004s	sys 0.009s
I20260709 13:09:54.788806 28229 raft_consensus.cc:740] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.790838 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
I20260709 13:09:54.791165 28227 tablet_bootstrap.cc:492] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:54.791126 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d" is_pre_election: true
I20260709 13:09:54.791030 28229 consensus_queue.cc:260] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.793042 28227 ts_tablet_manager.cc:1403] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.084s	user 0.016s	sys 0.010s
I20260709 13:09:54.793291 28228 tablet_bootstrap.cc:492] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
W20260709 13:09:54.794114 28001 leader_election.cc:343] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.799739 28229 ts_tablet_manager.cc:1434] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.005s
I20260709 13:09:54.801132 28229 tablet_bootstrap.cc:492] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:54.804291 28000 leader_election.cc:304] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d
I20260709 13:09:54.805142 28235 raft_consensus.cc:2749] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:54.806804 28227 raft_consensus.cc:359] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:54.807978 28227 raft_consensus.cc:385] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.808643 28235 raft_consensus.cc:493] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.809116 28235 raft_consensus.cc:515] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.814177 28228 tablet_bootstrap.cc:654] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.817176 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
I20260709 13:09:54.808524 28227 raft_consensus.cc:740] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
W20260709 13:09:54.819018 28001 leader_election.cc:343] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.822671 28235 leader_election.cc:290] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:54.825814 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d" is_pre_election: true
I20260709 13:09:54.825075 28227 consensus_queue.cc:260] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
W20260709 13:09:54.827920 28000 leader_election.cc:343] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.828519 28000 leader_election.cc:304] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d
I20260709 13:09:54.829051 28227 ts_tablet_manager.cc:1434] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:09:54.830947 28235 raft_consensus.cc:2749] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:54.857142 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d" is_pre_election: true
I20260709 13:09:54.857822 28185 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 0.
I20260709 13:09:54.858415 28229 tablet_bootstrap.cc:654] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.844077 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:54.858930 28036 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 0.
I20260709 13:09:54.842433 28227 tablet_bootstrap.cc:492] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:54.860965 28075 leader_election.cc:304] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:54.861908 28234 raft_consensus.cc:2804] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.862706 28234 raft_consensus.cc:493] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.865911 28234 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:54.876762 28000 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:41679, user_credentials={real_user=slave}} blocked reactor thread for 48844.3us
I20260709 13:09:54.880267 28229 tablet_bootstrap.cc:492] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:54.880903 28229 ts_tablet_manager.cc:1403] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.080s	user 0.008s	sys 0.010s
I20260709 13:09:54.877938 27932 catalog_manager.cc:5697] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 0 to 1, leader changed from <none> to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 1 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:54.882797 28234 raft_consensus.cc:515] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:54.884265 28229 raft_consensus.cc:359] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.885456 28229 raft_consensus.cc:385] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.885771 28229 raft_consensus.cc:740] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.886523 28229 consensus_queue.cc:260] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.887845 28227 tablet_bootstrap.cc:654] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.889400 28234 leader_election.cc:290] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:54.890134 28228 tablet_bootstrap.cc:492] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:54.890594 28228 ts_tablet_manager.cc:1403] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.098s	user 0.017s	sys 0.005s
I20260709 13:09:54.892802 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:54.893373 28036 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.893539 28228 raft_consensus.cc:359] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.894277 28228 raft_consensus.cc:385] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.894541 28228 raft_consensus.cc:740] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.895232 28228 consensus_queue.cc:260] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.896418 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:54.897070 28185 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.898403 28228 ts_tablet_manager.cc:1434] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:54.899396 28228 tablet_bootstrap.cc:492] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
I20260709 13:09:54.912866 28229 ts_tablet_manager.cc:1434] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.009s
I20260709 13:09:54.914040 28229 tablet_bootstrap.cc:492] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:54.910156 28185 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 1.
I20260709 13:09:54.922027 28075 leader_election.cc:304] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:54.923058 28036 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 1.
I20260709 13:09:54.926143 28264 raft_consensus.cc:2804] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.928160 28264 raft_consensus.cc:697] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:09:54.929546 28228 tablet_bootstrap.cc:654] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.929970 28264 consensus_queue.cc:237] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:54.935940 28234 raft_consensus.cc:493] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.936489 28234 raft_consensus.cc:515] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.937156 28229 tablet_bootstrap.cc:654] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.937273 28227 tablet_bootstrap.cc:492] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:54.937986 28227 ts_tablet_manager.cc:1403] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.106s	user 0.010s	sys 0.017s
I20260709 13:09:54.944584 28234 leader_election.cc:290] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:54.945919 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce59f69f404a4584bce3d7f4c5cd8476" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d" is_pre_election: true
I20260709 13:09:54.945955 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce59f69f404a4584bce3d7f4c5cd8476" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
W20260709 13:09:54.949527 28076 leader_election.cc:343] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:54.951876 28075 leader_election.cc:343] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.952455 28076 leader_election.cc:304] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15aee576f44a4602894736f69680811b; no voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:09:54.953835 28264 raft_consensus.cc:2749] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:54.954211 28228 tablet_bootstrap.cc:492] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:54.954816 28228 ts_tablet_manager.cc:1403] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.056s	user 0.007s	sys 0.014s
I20260709 13:09:54.956562 28227 raft_consensus.cc:359] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.957305 28227 raft_consensus.cc:385] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.957614 28227 raft_consensus.cc:740] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.957759 28228 raft_consensus.cc:359] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.958580 28228 raft_consensus.cc:385] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.958364 28227 consensus_queue.cc:260] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.958879 28228 raft_consensus.cc:740] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.960717 28227 ts_tablet_manager.cc:1434] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:09:54.961669 28227 tablet_bootstrap.cc:492] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:54.961750 28228 consensus_queue.cc:260] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.967788 28228 ts_tablet_manager.cc:1434] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:09:54.968995 28228 tablet_bootstrap.cc:492] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
I20260709 13:09:54.975581 28228 tablet_bootstrap.cc:654] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.976567 27931 catalog_manager.cc:5697] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 0 to 1, leader changed from <none> to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:54.978391 28264 raft_consensus.cc:493] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.982340 28264 raft_consensus.cc:515] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.984480 28264 leader_election.cc:290] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:54.978622 28229 tablet_bootstrap.cc:492] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:54.985158 28229 ts_tablet_manager.cc:1403] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.015s
I20260709 13:09:54.988284 28227 tablet_bootstrap.cc:654] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.988679 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:54.988602 28229 raft_consensus.cc:359] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.989416 28229 raft_consensus.cc:385] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.989393 28036 raft_consensus.cc:2468] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 0.
I20260709 13:09:54.989893 28229 raft_consensus.cc:740] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.981801 28228 tablet_bootstrap.cc:492] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:54.990491 28228 ts_tablet_manager.cc:1403] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.007s
I20260709 13:09:54.990584 28229 consensus_queue.cc:260] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.991701 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d" is_pre_election: true
I20260709 13:09:54.993263 28229 ts_tablet_manager.cc:1434] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:09:54.993588 28228 raft_consensus.cc:359] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:54.994253 28228 raft_consensus.cc:385] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.994194 28076 leader_election.cc:304] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:54.994740 28228 raft_consensus.cc:740] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.995325 28264 raft_consensus.cc:2804] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.995678 28264 raft_consensus.cc:493] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.996019 28264 raft_consensus.cc:3060] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:54.996093 28075 leader_election.cc:343] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.999322 28228 consensus_queue.cc:260] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.001706 28228 ts_tablet_manager.cc:1434] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:09:55.003033 28264 raft_consensus.cc:515] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.013237 28264 leader_election.cc:290] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:54.994239 28229 tablet_bootstrap.cc:492] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:55.015400 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:55.016269 28036 raft_consensus.cc:3060] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.017463 28228 tablet_bootstrap.cc:492] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
I20260709 13:09:55.017683 28261 raft_consensus.cc:493] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.018313 28261 raft_consensus.cc:515] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.020481 28229 tablet_bootstrap.cc:654] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:55.022660 28261 leader_election.cc:290] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:55.023206 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e5120e7f37845eb977a87e7eca3b017" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d" is_pre_election: true
I20260709 13:09:55.023902 28185 raft_consensus.cc:2468] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 0.
I20260709 13:09:55.024560 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e5120e7f37845eb977a87e7eca3b017" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
I20260709 13:09:55.025274 28000 leader_election.cc:304] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.025456 28111 raft_consensus.cc:2468] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 0.
I20260709 13:09:55.028625 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:55.030043 28228 tablet_bootstrap.cc:654] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:55.030977 28075 leader_election.cc:343] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:55.036304 28261 raft_consensus.cc:2804] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:55.041352 28261 raft_consensus.cc:493] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.041976 28261 raft_consensus.cc:3060] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.043386 28228 tablet_bootstrap.cc:492] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:55.044000 28228 ts_tablet_manager.cc:1403] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.009s
I20260709 13:09:55.046723 28228 raft_consensus.cc:359] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.047405 28228 raft_consensus.cc:385] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:55.047678 28228 raft_consensus.cc:740] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.048574 28228 consensus_queue.cc:260] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.058053 28228 ts_tablet_manager.cc:1434] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:09:55.057952 28036 raft_consensus.cc:2468] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 1.
I20260709 13:09:55.059386 28228 tablet_bootstrap.cc:492] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d: Bootstrap starting.
I20260709 13:09:55.059902 28076 leader_election.cc:304] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 20ca690f4dc24bc2a051124def8a124d
I20260709 13:09:55.061012 28264 raft_consensus.cc:2804] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:55.061463 28264 raft_consensus.cc:697] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:09:55.055685 28261 raft_consensus.cc:515] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.062279 28264 consensus_queue.cc:237] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.066362 28261 leader_election.cc:290] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 election: Requested vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:55.067502 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e5120e7f37845eb977a87e7eca3b017" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:55.068084 28111 raft_consensus.cc:3060] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.068377 28227 tablet_bootstrap.cc:492] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:55.068821 28227 ts_tablet_manager.cc:1403] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.107s	user 0.030s	sys 0.057s
I20260709 13:09:55.070807 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e5120e7f37845eb977a87e7eca3b017" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:55.071381 28185 raft_consensus.cc:3060] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.071734 28227 raft_consensus.cc:359] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.072458 28227 raft_consensus.cc:385] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:55.072780 28227 raft_consensus.cc:740] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.073465 28227 consensus_queue.cc:260] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.076391 28227 ts_tablet_manager.cc:1434] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:55.076581 28228 tablet_bootstrap.cc:654] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:55.077385 28227 tablet_bootstrap.cc:492] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:55.080762 28241 raft_consensus.cc:493] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.081231 28241 raft_consensus.cc:515] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.083235 28241 leader_election.cc:290] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.084539 28110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
I20260709 13:09:55.085214 28110 raft_consensus.cc:2468] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:55.085708 28227 tablet_bootstrap.cc:654] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:55.086426 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:55.086493 28150 leader_election.cc:304] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:55.089010 27933 catalog_manager.cc:5697] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 0 to 1, leader changed from <none> to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.098881 28241 raft_consensus.cc:2804] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:55.099267 28241 raft_consensus.cc:493] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.099566 28241 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.106516 28241 raft_consensus.cc:515] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.107971 28233 raft_consensus.cc:493] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.108477 28233 raft_consensus.cc:515] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.110764 28233 leader_election.cc:290] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.097795 28185 raft_consensus.cc:2468] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 1.
I20260709 13:09:55.112931 28035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:55.114807 28036 raft_consensus.cc:2468] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:55.115082 28035 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:55.116302 28000 leader_election.cc:304] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.116914 28241 leader_election.cc:290] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.117056 28261 raft_consensus.cc:2804] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:55.117525 28261 raft_consensus.cc:697] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 LEADER]: Becoming Leader. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:09:55.118031 28229 tablet_bootstrap.cc:492] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:55.118482 28229 ts_tablet_manager.cc:1403] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.124s	user 0.004s	sys 0.023s
I20260709 13:09:55.118319 28261 consensus_queue.cc:237] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.115597 28228 tablet_bootstrap.cc:492] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d: No bootstrap required, opened a new log
I20260709 13:09:55.120499 28228 ts_tablet_manager.cc:1403] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.006s
I20260709 13:09:55.119791 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
I20260709 13:09:55.124606 28109 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:55.125329 28150 leader_election.cc:304] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.130228 28111 raft_consensus.cc:2468] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 1.
I20260709 13:09:55.131769 28241 raft_consensus.cc:2804] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:55.133747 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:55.134016 28035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:55.134343 28109 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.134618 28035 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.133766 28241 raft_consensus.cc:493] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.136873 28241 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.132594 28233 raft_consensus.cc:493] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.138314 28229 raft_consensus.cc:359] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
W20260709 13:09:55.135695 28150 consensus_peers.cc:597] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d -> Peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Couldn't send request to peer 15aee576f44a4602894736f69680811b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.140781 28233 raft_consensus.cc:515] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.143266 28233 leader_election.cc:290] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.140885 28229 raft_consensus.cc:385] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:55.144632 28109 raft_consensus.cc:2468] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:55.146449 28229 raft_consensus.cc:740] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.147539 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce59f69f404a4584bce3d7f4c5cd8476" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:55.147589 28228 raft_consensus.cc:359] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.148377 28228 raft_consensus.cc:385] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:55.148639 28228 raft_consensus.cc:740] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.148861 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce59f69f404a4584bce3d7f4c5cd8476" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
I20260709 13:09:55.149444 28111 raft_consensus.cc:2468] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:55.149266 28228 consensus_queue.cc:260] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.151484 28150 leader_election.cc:304] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:55.152159 28228 ts_tablet_manager.cc:1434] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:09:55.152652 28150 leader_election.cc:304] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:55.153594 28233 raft_consensus.cc:2804] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:55.147380 28229 consensus_queue.cc:260] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.156580 28242 raft_consensus.cc:2804] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:55.156966 28242 raft_consensus.cc:493] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.157322 28242 raft_consensus.cc:3060] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.157378 28229 ts_tablet_manager.cc:1434] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.003s
I20260709 13:09:55.161170 28229 tablet_bootstrap.cc:492] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:55.161660 28227 tablet_bootstrap.cc:492] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:55.162071 28227 ts_tablet_manager.cc:1403] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.085s	user 0.020s	sys 0.026s
W20260709 13:09:55.162710 28150 leader_election.cc:343] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:55.166996 28227 raft_consensus.cc:359] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.167657 28227 raft_consensus.cc:385] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:55.167928 28227 raft_consensus.cc:740] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.168648 28227 consensus_queue.cc:260] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.170923 28227 ts_tablet_manager.cc:1434] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:09:55.170810 27933 catalog_manager.cc:5697] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.172150 28233 raft_consensus.cc:697] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:55.173478 28227 tablet_bootstrap.cc:492] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8: Bootstrap starting.
I20260709 13:09:55.179878 28233 consensus_queue.cc:237] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.196234 28227 tablet_bootstrap.cc:654] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:55.187556 28035 raft_consensus.cc:2468] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
W20260709 13:09:55.179864 28150 consensus_peers.cc:597] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d -> Peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Couldn't send request to peer 15aee576f44a4602894736f69680811b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:55.192456 28241 raft_consensus.cc:515] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.193279 28229 tablet_bootstrap.cc:654] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:55.208320 28241 leader_election.cc:290] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.193586 28242 raft_consensus.cc:515] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.210858 28242 leader_election.cc:290] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.210690 28034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:55.211455 28034 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.212608 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:55.213217 28111 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.214702 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce59f69f404a4584bce3d7f4c5cd8476" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:55.215123 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce59f69f404a4584bce3d7f4c5cd8476" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:55.215682 28109 raft_consensus.cc:3060] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.216264 28242 consensus_queue.cc:1048] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.216969 28227 tablet_bootstrap.cc:492] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8: No bootstrap required, opened a new log
I20260709 13:09:55.217370 28227 ts_tablet_manager.cc:1403] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent bootstrapping tablet: real 0.044s	user 0.004s	sys 0.011s
I20260709 13:09:55.217747 28242 consensus_queue.cc:1048] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.215265 28036 raft_consensus.cc:3060] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.219180 27933 catalog_manager.cc:5697] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 0 to 1, leader changed from <none> to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 1 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:55.224259 28227 raft_consensus.cc:359] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.224675 28111 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:55.225021 28227 raft_consensus.cc:385] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:55.225306 28227 raft_consensus.cc:740] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.227295 28229 tablet_bootstrap.cc:492] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:55.227754 28229 ts_tablet_manager.cc:1403] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.067s	user 0.004s	sys 0.007s
I20260709 13:09:55.233135 28150 leader_election.cc:304] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:55.234040 28233 raft_consensus.cc:2804] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:55.234622 28233 raft_consensus.cc:697] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:55.235433 28233 consensus_queue.cc:237] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.236483 28229 raft_consensus.cc:359] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.237200 28229 raft_consensus.cc:385] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:55.237470 28229 raft_consensus.cc:740] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.238293 28242 raft_consensus.cc:493] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.238440 28229 consensus_queue.cc:260] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.238776 28242 raft_consensus.cc:515] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.239667 28109 raft_consensus.cc:2468] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:55.240556 28229 ts_tablet_manager.cc:1434] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:09:55.240972 28227 consensus_queue.cc:260] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.243932 28034 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:55.251129 28229 tablet_bootstrap.cc:492] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b: Bootstrap starting.
I20260709 13:09:55.254014 28242 leader_election.cc:290] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.256242 28034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:55.257814 28034 raft_consensus.cc:2393] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20ca690f4dc24bc2a051124def8a124d in current term 1: Already voted for candidate 15aee576f44a4602894736f69680811b in this term.
I20260709 13:09:55.256546 28150 leader_election.cc:304] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:55.261458 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
I20260709 13:09:55.263485 28227 ts_tablet_manager.cc:1434] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.004s
I20260709 13:09:55.264241 28150 leader_election.cc:304] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d; no voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:09:55.264957 28242 raft_consensus.cc:2804] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:55.262771 28036 raft_consensus.cc:2468] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:55.268181 28241 raft_consensus.cc:3060] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.278326 28229 tablet_bootstrap.cc:654] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:55.281435 28241 raft_consensus.cc:2749] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:55.281858 28242 raft_consensus.cc:697] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:55.284399 28242 consensus_queue.cc:237] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
W20260709 13:09:55.291448 27981 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:55.293573 27930 catalog_manager.cc:5697] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 0 to 1, leader changed from <none> to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 1 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:55.317247 28229 tablet_bootstrap.cc:492] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b: No bootstrap required, opened a new log
I20260709 13:09:55.317736 28229 ts_tablet_manager.cc:1403] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.000s
I20260709 13:09:55.320451 28229 raft_consensus.cc:359] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.321161 28229 raft_consensus.cc:385] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:55.321447 28229 raft_consensus.cc:740] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Initialized, Role: FOLLOWER
I20260709 13:09:55.322080 28229 consensus_queue.cc:260] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.321888 27931 catalog_manager.cc:5697] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 0 to 1, leader changed from <none> to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 1 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:55.324501 28229 ts_tablet_manager.cc:1434] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.002s
I20260709 13:09:55.330845 28264 consensus_queue.cc:1048] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.356241 28264 consensus_queue.cc:1048] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.366626 28295 raft_consensus.cc:493] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.367165 28295 raft_consensus.cc:515] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.369035 28295 leader_election.cc:290] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:55.370599 28035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:55.370613 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ca690f4dc24bc2a051124def8a124d" is_pre_election: true
I20260709 13:09:55.372485 28076 leader_election.cc:304] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15aee576f44a4602894736f69680811b; no voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:09:55.373272 28295 raft_consensus.cc:2749] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:55.382707 28242 raft_consensus.cc:493] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.383204 28242 raft_consensus.cc:515] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.385070 28242 leader_election.cc:290] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.385962 28035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" is_pre_election: true
I20260709 13:09:55.386301 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b" is_pre_election: true
I20260709 13:09:55.386655 28035 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:55.386878 28109 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 0.
I20260709 13:09:55.387780 28150 leader_election.cc:304] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.388545 28242 raft_consensus.cc:2804] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:55.388903 28242 raft_consensus.cc:493] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.389271 28242 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.393635 28242 raft_consensus.cc:515] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.395352 28242 leader_election.cc:290] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.396263 28035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:55.396919 28035 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.396788 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:55.397483 28109 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.401374 28035 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:55.402521 28150 leader_election.cc:304] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.403378 28242 raft_consensus.cc:2804] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:55.404176 28242 raft_consensus.cc:697] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:55.404598 28109 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 1.
I20260709 13:09:55.405035 28242 consensus_queue.cc:237] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.413892 27931 catalog_manager.cc:5697] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 0 to 1, leader changed from <none> to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 1 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:55.531493 28295 consensus_queue.cc:1048] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.543200 28234 consensus_queue.cc:1048] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.584515 28242 consensus_queue.cc:1048] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.588848 28111 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.601325 28284 consensus_queue.cc:1048] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.641665 28111 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d"
dest_uuid: "15aee576f44a4602894736f69680811b"
mode: GRACEFUL
new_leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:41202
I20260709 13:09:55.642366 28111 raft_consensus.cc:606] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Received request to transfer leadership to TS e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:09:55.655859 28185 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
mode: GRACEFUL
new_leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:49190
I20260709 13:09:55.656524 28185 raft_consensus.cc:606] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Received request to transfer leadership to TS e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:09:55.661881 28185 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
mode: GRACEFUL
new_leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:49190
I20260709 13:09:55.662500 28185 raft_consensus.cc:606] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Received request to transfer leadership to TS e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:09:55.670709 28185 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
mode: GRACEFUL
new_leader_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:49190
I20260709 13:09:55.671299 28185 raft_consensus.cc:606] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Received request to transfer leadership to TS 15aee576f44a4602894736f69680811b
I20260709 13:09:55.675948 28261 consensus_queue.cc:1048] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.679330 28184 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70e6679c989543a4a60743fbf5628180"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
mode: GRACEFUL
new_leader_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:49190
I20260709 13:09:55.680140 28184 raft_consensus.cc:606] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Received request to transfer leadership to TS 15aee576f44a4602894736f69680811b
I20260709 13:09:55.681604 28111 raft_consensus.cc:3060] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.681625 28306 consensus_queue.cc:1048] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.695685 28185 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
mode: GRACEFUL
new_leader_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:49190
I20260709 13:09:55.696324 28185 raft_consensus.cc:606] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Received request to transfer leadership to TS 15aee576f44a4602894736f69680811b
I20260709 13:09:55.717043 28284 consensus_queue.cc:1048] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.723150 28284 raft_consensus.cc:993] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d: : Instructing follower 15aee576f44a4602894736f69680811b to start an election
I20260709 13:09:55.723631 28284 raft_consensus.cc:1081] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Signalling peer 15aee576f44a4602894736f69680811b to start an election
I20260709 13:09:55.726214 28110 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789"
dest_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:41182
I20260709 13:09:55.726846 28110 raft_consensus.cc:493] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.727312 28110 raft_consensus.cc:3060] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.741554 28110 raft_consensus.cc:515] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.759778 28035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:55.760079 28110 leader_election.cc:290] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 2 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:55.760546 28035 raft_consensus.cc:3060] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.756853 28284 consensus_queue.cc:1048] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:09:55.765297 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6faee2f44c7e44db9f459461b9279789" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:55.765960 28185 raft_consensus.cc:3055] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.766315 28185 raft_consensus.cc:740] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:55.766587 28285 raft_consensus.cc:993] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d: : Instructing follower e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:09:55.767047 28285 raft_consensus.cc:1081] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Signalling peer e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:09:55.769186 28036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:56718
I20260709 13:09:55.769760 28036 raft_consensus.cc:493] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.770174 28036 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.776347 28185 consensus_queue.cc:260] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.777842 28185 raft_consensus.cc:3060] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.778465 28036 raft_consensus.cc:515] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.781682 28110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:55.782361 28110 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.783989 28184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:55.785181 28184 raft_consensus.cc:3055] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.785549 28184 raft_consensus.cc:740] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:55.786450 28184 consensus_queue.cc:260] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.787668 28184 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.775764 28035 raft_consensus.cc:2468] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 2.
I20260709 13:09:55.789634 28033 raft_consensus.cc:1240] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Rejecting Update request from peer 20ca690f4dc24bc2a051124def8a124d for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:55.793262 28076 leader_election.cc:304] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.794227 28264 raft_consensus.cc:2804] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.794756 28264 raft_consensus.cc:697] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:09:55.795641 28264 consensus_queue.cc:237] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.785007 28036 leader_election.cc:290] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 2 election: Requested vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:55.801683 28185 raft_consensus.cc:2468] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 2.
I20260709 13:09:55.808264 28184 raft_consensus.cc:2468] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 2.
I20260709 13:09:55.809580 28000 leader_election.cc:304] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.810725 28311 raft_consensus.cc:2804] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.811918 28311 raft_consensus.cc:697] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Becoming Leader. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:09:55.813450 28311 consensus_queue.cc:237] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.818491 27933 catalog_manager.cc:5697] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 1 to 2, leader changed from 20ca690f4dc24bc2a051124def8a124d (127.27.63.67) to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 2 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.820876 28110 raft_consensus.cc:2468] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 2.
I20260709 13:09:55.821703 28234 raft_consensus.cc:993] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b: : Instructing follower e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:09:55.822173 28264 raft_consensus.cc:1081] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Signalling peer e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:09:55.826756 28033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:56728
I20260709 13:09:55.827387 28033 raft_consensus.cc:493] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.827790 28033 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.829214 28284 raft_consensus.cc:993] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d: : Instructing follower e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:09:55.829679 28242 raft_consensus.cc:1081] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Signalling peer e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:09:55.831861 28036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:56718
I20260709 13:09:55.832602 28036 raft_consensus.cc:493] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.833021 28036 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.838156 28033 raft_consensus.cc:515] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:55.840206 28036 raft_consensus.cc:515] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.841287 28184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:55.841900 28184 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.842519 28110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:55.843140 28110 raft_consensus.cc:3055] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.843500 28110 raft_consensus.cc:740] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:09:55.843684 28036 leader_election.cc:290] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 2 election: Requested vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:55.844864 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:55.841914 28033 leader_election.cc:290] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 2 election: Requested vote from peers 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:55.845505 28185 raft_consensus.cc:3055] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.845916 28185 raft_consensus.cc:740] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:55.846797 28185 consensus_queue.cc:260] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.848476 28111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:55.848333 28110 consensus_queue.cc:260] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:55.849054 28111 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.849522 28110 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.851382 28184 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 2.
I20260709 13:09:55.852764 28000 leader_election.cc:304] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.853821 28311 raft_consensus.cc:2804] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.854326 28311 raft_consensus.cc:697] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Becoming Leader. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:09:55.854712 28241 consensus_queue.cc:1048] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.855237 28311 consensus_queue.cc:237] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:55.856838 27933 catalog_manager.cc:5697] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 reported cstate change: term changed from 1 to 2, leader changed from 20ca690f4dc24bc2a051124def8a124d (127.27.63.67) to e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.860304 28185 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.861652 28111 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 2.
I20260709 13:09:55.864135 28001 leader_election.cc:304] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.864995 28306 raft_consensus.cc:2804] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.865473 28306 raft_consensus.cc:697] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Becoming Leader. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:09:55.866420 28306 consensus_queue.cc:237] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.867164 28185 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 2.
I20260709 13:09:55.875242 28241 consensus_queue.cc:1048] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.881652 27931 catalog_manager.cc:5697] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 reported cstate change: term changed from 1 to 2, leader changed from 20ca690f4dc24bc2a051124def8a124d (127.27.63.67) to e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.883955 28110 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 2.
I20260709 13:09:55.888254 27931 catalog_manager.cc:5697] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 reported cstate change: term changed from 1 to 2, leader changed from 15aee576f44a4602894736f69680811b (127.27.63.66) to e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.943456 28241 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.950250 28285 raft_consensus.cc:993] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d: : Instructing follower 15aee576f44a4602894736f69680811b to start an election
I20260709 13:09:55.950711 28285 raft_consensus.cc:1081] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Signalling peer 15aee576f44a4602894736f69680811b to start an election
I20260709 13:09:55.953660 28109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e"
dest_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:41182
I20260709 13:09:55.954288 28109 raft_consensus.cc:493] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.954725 28109 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.957588 28241 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.963322 28109 raft_consensus.cc:515] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.965310 28110 raft_consensus.cc:1240] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Rejecting Update request from peer 20ca690f4dc24bc2a051124def8a124d for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:55.965886 28109 leader_election.cc:290] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 2 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:55.967475 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:55.967803 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:55.968086 28036 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.968421 28185 raft_consensus.cc:3055] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.968783 28185 raft_consensus.cc:740] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:55.969161 28241 consensus_queue.cc:1059] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.976068 28036 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 2.
I20260709 13:09:55.976020 28185 consensus_queue.cc:260] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.977540 28076 leader_election.cc:304] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:55.978438 28264 raft_consensus.cc:2804] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.978933 28264 raft_consensus.cc:697] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:09:55.979311 28185 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.979725 28264 consensus_queue.cc:237] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:55.986622 27933 catalog_manager.cc:5697] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 1 to 2, leader changed from 20ca690f4dc24bc2a051124def8a124d (127.27.63.67) to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 2 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.987916 28185 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 2.
W20260709 13:09:55.988494 28241 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:09:56.124816 28285 raft_consensus.cc:993] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d: : Instructing follower 15aee576f44a4602894736f69680811b to start an election
I20260709 13:09:56.125309 28241 raft_consensus.cc:1081] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Signalling peer 15aee576f44a4602894736f69680811b to start an election
I20260709 13:09:56.126782 28110 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70e6679c989543a4a60743fbf5628180"
dest_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:41182
I20260709 13:09:56.127300 28110 raft_consensus.cc:493] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:56.127594 28110 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.132335 28110 raft_consensus.cc:515] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:56.133988 28110 leader_election.cc:290] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 2 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:56.135083 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:56.135072 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:56.135843 28036 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.135891 28185 raft_consensus.cc:3055] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:56.136433 28185 raft_consensus.cc:740] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:56.137265 28185 consensus_queue.cc:260] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:56.138501 28185 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.142900 28036 raft_consensus.cc:2468] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 2.
I20260709 13:09:56.144299 28076 leader_election.cc:304] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:56.145112 28185 raft_consensus.cc:2468] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 2.
I20260709 13:09:56.145133 28264 raft_consensus.cc:2804] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:56.145728 28264 raft_consensus.cc:697] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:09:56.146512 28264 consensus_queue.cc:237] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:56.154325 27933 catalog_manager.cc:5697] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 1 to 2, leader changed from 20ca690f4dc24bc2a051124def8a124d (127.27.63.67) to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 2 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:56.197873 28185 raft_consensus.cc:1275] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.199160 28264 consensus_queue.cc:1048] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.207221 28036 raft_consensus.cc:1275] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.208771 28295 consensus_queue.cc:1048] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.278435 28185 raft_consensus.cc:1275] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.279875 28306 consensus_queue.cc:1048] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.289258 28110 raft_consensus.cc:1275] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.296809 28306 consensus_queue.cc:1048] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:56.298805 28110 raft_consensus.cc:1275] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.301182 28185 raft_consensus.cc:1275] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.303460 28311 consensus_queue.cc:1048] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:56.303558 28306 consensus_queue.cc:1048] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:56.314451 28184 raft_consensus.cc:1275] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.315972 28261 consensus_queue.cc:1048] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.342257 28109 raft_consensus.cc:1275] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.343593 28306 consensus_queue.cc:1048] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.514258 28185 raft_consensus.cc:1275] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.514747 28036 raft_consensus.cc:1275] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.515558 28295 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.516244 28264 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.547374 28185 raft_consensus.cc:1275] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.548513 28264 consensus_queue.cc:1048] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.556003 28036 raft_consensus.cc:1275] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.557617 28295 consensus_queue.cc:1048] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:58.740409 27901 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:09:58.741214 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:58.763881 28109 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e"
dest_uuid: "15aee576f44a4602894736f69680811b"
mode: GRACEFUL
new_leader_uuid: "20ca690f4dc24bc2a051124def8a124d"
 from {username='slave'} at 127.0.0.1:41214
I20260709 13:09:58.764456 28109 raft_consensus.cc:606] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Received request to transfer leadership to TS 20ca690f4dc24bc2a051124def8a124d
I20260709 13:09:58.774885 28036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
mode: GRACEFUL
new_leader_uuid: "20ca690f4dc24bc2a051124def8a124d"
 from {username='slave'} at 127.0.0.1:56734
I20260709 13:09:58.775506 28036 raft_consensus.cc:606] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Received request to transfer leadership to TS 20ca690f4dc24bc2a051124def8a124d
I20260709 13:09:58.776551 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:58.776980 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:58.974318 28306 raft_consensus.cc:993] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8: : Instructing follower 20ca690f4dc24bc2a051124def8a124d to start an election
I20260709 13:09:58.974808 28320 raft_consensus.cc:1081] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Signalling peer 20ca690f4dc24bc2a051124def8a124d to start an election
I20260709 13:09:58.976212 28185 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
 from {username='slave'} at 127.0.0.1:49182
I20260709 13:09:58.976678 28185 raft_consensus.cc:493] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:58.976971 28185 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.982115 28185 raft_consensus.cc:515] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:58.983745 28185 leader_election.cc:290] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:58.984460 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:58.984833 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:58.985155 28036 raft_consensus.cc:3055] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:58.985428 28109 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.985543 28036 raft_consensus.cc:740] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:09:58.986277 28036 consensus_queue.cc:260] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:58.987154 28036 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.990126 28109 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 3.
I20260709 13:09:58.991357 28150 leader_election.cc:304] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:58.992081 28241 raft_consensus.cc:2804] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:58.992390 28036 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 3.
I20260709 13:09:58.993407 28241 raft_consensus.cc:697] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:58.994158 28321 raft_consensus.cc:993] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b: : Instructing follower 20ca690f4dc24bc2a051124def8a124d to start an election
I20260709 13:09:58.994647 28295 raft_consensus.cc:1081] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Signalling peer 20ca690f4dc24bc2a051124def8a124d to start an election
I20260709 13:09:58.994295 28241 consensus_queue.cc:237] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:09:58.996090 28185 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
 from {username='slave'} at 127.0.0.1:49186
I20260709 13:09:58.996587 28185 raft_consensus.cc:493] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:58.996855 28185 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:59.001459 28185 raft_consensus.cc:515] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:59.003118 28185 leader_election.cc:290] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:09:59.003978 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:59.004469 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:09:59.004639 28036 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:59.005029 28109 raft_consensus.cc:3055] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:59.005340 28109 raft_consensus.cc:740] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:09:59.006074 28109 consensus_queue.cc:260] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:59.007300 28109 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:59.007622 27933 catalog_manager.cc:5697] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 2 to 3, leader changed from e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65) to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 3 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:59.011246 28036 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 3.
I20260709 13:09:59.012380 28150 leader_election.cc:304] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:09:59.013243 28241 raft_consensus.cc:2804] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:59.013854 28241 raft_consensus.cc:697] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:09:59.014159 28109 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 3.
I20260709 13:09:59.014609 28241 consensus_queue.cc:237] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:59.024834 27933 catalog_manager.cc:5697] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 2 to 3, leader changed from 15aee576f44a4602894736f69680811b (127.27.63.66) to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 3 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.496636 28109 raft_consensus.cc:1275] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Refusing update from remote peer 20ca690f4dc24bc2a051124def8a124d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:59.498021 28241 consensus_queue.cc:1048] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.506678 28036 raft_consensus.cc:1275] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Refusing update from remote peer 20ca690f4dc24bc2a051124def8a124d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:59.508391 28325 consensus_queue.cc:1048] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.572359 28109 raft_consensus.cc:1275] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Refusing update from remote peer 20ca690f4dc24bc2a051124def8a124d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:59.573490 28325 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.583971 28036 raft_consensus.cc:1275] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Refusing update from remote peer 20ca690f4dc24bc2a051124def8a124d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:59.585407 28241 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.777664 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:59.786895 28185 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
mode: GRACEFUL
new_leader_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:49190
I20260709 13:09:59.787348 28185 raft_consensus.cc:606] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Received request to transfer leadership to TS 15aee576f44a4602894736f69680811b
I20260709 13:09:59.800683 28036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
mode: GRACEFUL
new_leader_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:56750
I20260709 13:09:59.801144 28036 raft_consensus.cc:606] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Received request to transfer leadership to TS 15aee576f44a4602894736f69680811b
I20260709 13:09:59.805611 28036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
mode: GRACEFUL
new_leader_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:56750
I20260709 13:09:59.806048 28036 raft_consensus.cc:606] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Received request to transfer leadership to TS 15aee576f44a4602894736f69680811b
I20260709 13:09:59.810503 28036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e5120e7f37845eb977a87e7eca3b017"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
mode: GRACEFUL
new_leader_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:56750
I20260709 13:09:59.810942 28036 raft_consensus.cc:606] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 LEADER]: Received request to transfer leadership to TS 15aee576f44a4602894736f69680811b
I20260709 13:09:59.821843 28185 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
mode: GRACEFUL
new_leader_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:49190
I20260709 13:09:59.822358 28185 raft_consensus.cc:606] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Received request to transfer leadership to TS 15aee576f44a4602894736f69680811b
I20260709 13:09:59.955276 28320 raft_consensus.cc:993] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: : Instructing follower 15aee576f44a4602894736f69680811b to start an election
I20260709 13:09:59.955688 28342 raft_consensus.cc:1081] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Signalling peer 15aee576f44a4602894736f69680811b to start an election
I20260709 13:09:59.957087 28109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8"
dest_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:41192
I20260709 13:09:59.957662 28109 raft_consensus.cc:493] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:59.957957 28109 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:59.962412 28109 raft_consensus.cc:515] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:59.964454 28109 leader_election.cc:290] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 3 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:09:59.965289 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:09:59.965463 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:09:59.965879 28036 raft_consensus.cc:3055] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:59.965883 28185 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:59.966253 28036 raft_consensus.cc:740] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:09:59.966868 28036 consensus_queue.cc:260] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:59.967698 28036 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:59.969853 28185 raft_consensus.cc:2468] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 3.
I20260709 13:09:59.970847 28075 leader_election.cc:304] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:09:59.971698 28036 raft_consensus.cc:2468] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 3.
I20260709 13:09:59.971748 28295 raft_consensus.cc:2804] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:59.973006 28295 raft_consensus.cc:697] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:09:59.973701 28295 consensus_queue.cc:237] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:09:59.981052 27933 catalog_manager.cc:5697] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 2 to 3, leader changed from e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65) to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 3 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:00.092027 28241 raft_consensus.cc:993] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d: : Instructing follower 15aee576f44a4602894736f69680811b to start an election
I20260709 13:10:00.092617 28325 raft_consensus.cc:1081] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Signalling peer 15aee576f44a4602894736f69680811b to start an election
I20260709 13:10:00.094004 28109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d"
dest_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:41182
I20260709 13:10:00.094456 28109 raft_consensus.cc:493] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:00.094739 28109 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.099550 28109 raft_consensus.cc:515] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:10:00.101312 28109 leader_election.cc:290] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 4 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:10:00.102010 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:10:00.102303 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4ae96b1b064518844fc56e9296677d" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:10:00.102552 28036 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.102644 28320 raft_consensus.cc:993] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8: : Instructing follower 15aee576f44a4602894736f69680811b to start an election
I20260709 13:10:00.102824 28185 raft_consensus.cc:3055] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:00.103174 28185 raft_consensus.cc:740] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:10:00.103266 28320 raft_consensus.cc:1081] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Signalling peer 15aee576f44a4602894736f69680811b to start an election
I20260709 13:10:00.103993 28185 consensus_queue.cc:260] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:10:00.104739 28109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081"
dest_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:41192
I20260709 13:10:00.105105 28185 raft_consensus.cc:3060] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.105194 28109 raft_consensus.cc:493] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:00.105579 28109 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:00.108443 28036 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 4.
I20260709 13:10:00.109549 28076 leader_election.cc:304] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:10:00.109870 28109 raft_consensus.cc:515] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.110317 28295 raft_consensus.cc:2804] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:00.110772 28295 raft_consensus.cc:697] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 4 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:00.110901 28185 raft_consensus.cc:2468] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 4.
I20260709 13:10:00.111555 28295 consensus_queue.cc:237] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:10:00.112322 28109 leader_election.cc:290] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 3 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:10:00.113327 28184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:10:00.113925 28184 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:00.113449 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:10:00.115993 28036 raft_consensus.cc:3055] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:00.116360 28036 raft_consensus.cc:740] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:10:00.117000 28036 consensus_queue.cc:260] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.118062 28036 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:00.118029 27933 catalog_manager.cc:5697] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 3 to 4, leader changed from 20ca690f4dc24bc2a051124def8a124d (127.27.63.67) to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 4 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:00.123029 28184 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 3.
I20260709 13:10:00.123263 28036 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 3.
I20260709 13:10:00.124312 28075 leader_election.cc:304] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:10:00.125335 28295 raft_consensus.cc:2804] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:00.125811 28295 raft_consensus.cc:697] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:00.126572 28295 consensus_queue.cc:237] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.132759 27933 catalog_manager.cc:5697] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 2 to 3, leader changed from e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65) to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 3 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:00.156196 28241 raft_consensus.cc:993] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d: : Instructing follower 15aee576f44a4602894736f69680811b to start an election
I20260709 13:10:00.156693 28325 raft_consensus.cc:1081] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Signalling peer 15aee576f44a4602894736f69680811b to start an election
I20260709 13:10:00.156770 28342 raft_consensus.cc:993] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8: : Instructing follower 15aee576f44a4602894736f69680811b to start an election
I20260709 13:10:00.157263 28320 raft_consensus.cc:1081] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 LEADER]: Signalling peer 15aee576f44a4602894736f69680811b to start an election
I20260709 13:10:00.158095 28110 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e"
dest_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:41182
I20260709 13:10:00.158655 28110 raft_consensus.cc:493] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:00.158625 28109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e5120e7f37845eb977a87e7eca3b017"
dest_uuid: "15aee576f44a4602894736f69680811b"
 from {username='slave'} at 127.0.0.1:41192
I20260709 13:10:00.159073 28110 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.159160 28109 raft_consensus.cc:493] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:00.159523 28109 raft_consensus.cc:3060] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:00.163884 28110 raft_consensus.cc:515] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.163964 28109 raft_consensus.cc:515] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.165650 28109 leader_election.cc:290] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 2 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:10:00.165759 28110 leader_election.cc:290] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 4 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:10:00.166462 28036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:10:00.166992 28036 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.166944 28184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e5120e7f37845eb977a87e7eca3b017" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:10:00.167292 28033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e5120e7f37845eb977a87e7eca3b017" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:10:00.167619 28184 raft_consensus.cc:3060] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:00.167834 28033 raft_consensus.cc:3055] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:00.168184 28033 raft_consensus.cc:740] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:10:00.168792 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "15aee576f44a4602894736f69680811b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:10:00.169338 28185 raft_consensus.cc:3055] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:00.168943 28033 consensus_queue.cc:260] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.169682 28185 raft_consensus.cc:740] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:10:00.170269 28033 raft_consensus.cc:3060] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:00.170593 28185 consensus_queue.cc:260] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.171801 28185 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.173458 28036 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 4.
I20260709 13:10:00.174520 28076 leader_election.cc:304] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:10:00.174610 28184 raft_consensus.cc:2468] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 2.
I20260709 13:10:00.175374 28295 raft_consensus.cc:2804] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:00.175788 28075 leader_election.cc:304] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:10:00.175838 28295 raft_consensus.cc:697] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 4 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:00.177322 28343 raft_consensus.cc:2804] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:00.177815 28185 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 4.
I20260709 13:10:00.177464 28295 consensus_queue.cc:237] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.178364 28033 raft_consensus.cc:2468] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15aee576f44a4602894736f69680811b in term 2.
I20260709 13:10:00.180104 28343 raft_consensus.cc:697] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Becoming Leader. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:00.180863 28343 consensus_queue.cc:237] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:00.184535 27933 catalog_manager.cc:5697] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 3 to 4, leader changed from 20ca690f4dc24bc2a051124def8a124d (127.27.63.67) to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 4 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:00.194380 27933 catalog_manager.cc:5697] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b reported cstate change: term changed from 1 to 2, leader changed from e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65) to 15aee576f44a4602894736f69680811b (127.27.63.66). New cstate: current_term: 2 leader_uuid: "15aee576f44a4602894736f69680811b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:00.502295 28033 raft_consensus.cc:1275] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:00.503546 28295 consensus_queue.cc:1048] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:00.514394 28185 raft_consensus.cc:1275] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:00.516070 28295 consensus_queue.cc:1048] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:00.590461 28033 raft_consensus.cc:1275] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:00.591926 28295 consensus_queue.cc:1048] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:00.602047 28185 raft_consensus.cc:1275] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:00.603909 28343 consensus_queue.cc:1048] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:00.612782 28185 raft_consensus.cc:1275] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:00.613906 28295 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:00.623643 28033 raft_consensus.cc:1275] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:00.625080 28343 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:00.658253 28033 raft_consensus.cc:1275] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:00.659548 28343 consensus_queue.cc:1048] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:00.668119 28185 raft_consensus.cc:1275] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:00.669454 28343 consensus_queue.cc:1048] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:00.705931 28033 raft_consensus.cc:1275] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:00.707218 28343 consensus_queue.cc:1048] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:00.715699 28185 raft_consensus.cc:1275] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Refusing update from remote peer 15aee576f44a4602894736f69680811b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:00.717175 28295 consensus_queue.cc:1048] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:02.945581 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:02.952600 28110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8"
dest_uuid: "15aee576f44a4602894736f69680811b"
mode: GRACEFUL
new_leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:41214
I20260709 13:10:02.953282 28110 raft_consensus.cc:606] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Received request to transfer leadership to TS e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:02.955746 28110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8"
dest_uuid: "15aee576f44a4602894736f69680811b"
mode: GRACEFUL
new_leader_uuid: "20ca690f4dc24bc2a051124def8a124d"
 from {username='slave'} at 127.0.0.1:41214
I20260709 13:10:02.956235 28110 raft_consensus.cc:606] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Received request to transfer leadership to TS 20ca690f4dc24bc2a051124def8a124d
I20260709 13:10:02.958458 28110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70e6679c989543a4a60743fbf5628180"
dest_uuid: "15aee576f44a4602894736f69680811b"
mode: GRACEFUL
new_leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:41214
I20260709 13:10:02.958954 28110 raft_consensus.cc:606] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Received request to transfer leadership to TS e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:02.961020 28110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081"
dest_uuid: "15aee576f44a4602894736f69680811b"
mode: GRACEFUL
new_leader_uuid: "20ca690f4dc24bc2a051124def8a124d"
 from {username='slave'} at 127.0.0.1:41214
I20260709 13:10:02.961475 28110 raft_consensus.cc:606] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Received request to transfer leadership to TS 20ca690f4dc24bc2a051124def8a124d
I20260709 13:10:02.963474 28110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e"
dest_uuid: "15aee576f44a4602894736f69680811b"
mode: GRACEFUL
new_leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:41214
I20260709 13:10:02.963919 28110 raft_consensus.cc:606] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 4 LEADER]: Received request to transfer leadership to TS e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:02.964951 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:02.965396 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:03.046375 28366 raft_consensus.cc:993] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b: : Instructing follower 20ca690f4dc24bc2a051124def8a124d to start an election
I20260709 13:10:03.046928 28366 raft_consensus.cc:1081] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Signalling peer 20ca690f4dc24bc2a051124def8a124d to start an election
I20260709 13:10:03.048466 28185 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
 from {username='slave'} at 127.0.0.1:49186
I20260709 13:10:03.048950 28185 raft_consensus.cc:493] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.049240 28185 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.054253 28185 raft_consensus.cc:515] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.055876 28185 leader_election.cc:290] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 4 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:10:03.057307 28110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:10:03.057157 28033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db7d5960e184238a5cecc537f2aedd8" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:10:03.058005 28110 raft_consensus.cc:3055] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:03.058105 28033 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.058487 28110 raft_consensus.cc:740] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:03.059551 28110 consensus_queue.cc:260] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.060952 28110 raft_consensus.cc:3060] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.065877 28033 raft_consensus.cc:2468] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 4.
I20260709 13:10:03.067389 28150 leader_election.cc:304] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:10:03.068266 28110 raft_consensus.cc:2468] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 4.
I20260709 13:10:03.068325 28325 raft_consensus.cc:2804] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:03.069617 28325 raft_consensus.cc:697] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 4 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:10:03.070597 28325 consensus_queue.cc:237] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.072388 28367 raft_consensus.cc:993] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b: : Instructing follower 20ca690f4dc24bc2a051124def8a124d to start an election
I20260709 13:10:03.072889 28366 raft_consensus.cc:1081] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Signalling peer 20ca690f4dc24bc2a051124def8a124d to start an election
I20260709 13:10:03.074478 28185 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081"
dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
 from {username='slave'} at 127.0.0.1:49186
I20260709 13:10:03.074936 28185 raft_consensus.cc:493] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.075304 28185 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.081308 28185 raft_consensus.cc:515] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.083199 28185 leader_election.cc:290] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 4 election: Requested vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
I20260709 13:10:03.084195 28110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:10:03.083948 27930 catalog_manager.cc:5697] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 3 to 4, leader changed from 15aee576f44a4602894736f69680811b (127.27.63.66) to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 4 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:03.084761 28110 raft_consensus.cc:3055] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:03.085081 28110 raft_consensus.cc:740] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:03.085883 28110 consensus_queue.cc:260] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.087193 28110 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.088634 28033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "787dd4bb42b548c39e14ada836c30081" candidate_uuid: "20ca690f4dc24bc2a051124def8a124d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
I20260709 13:10:03.089468 28033 raft_consensus.cc:3060] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.093475 28110 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 4.
I20260709 13:10:03.094872 28150 leader_election.cc:304] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15aee576f44a4602894736f69680811b, 20ca690f4dc24bc2a051124def8a124d; no voters: 
I20260709 13:10:03.095580 28325 raft_consensus.cc:2804] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:03.096422 28325 raft_consensus.cc:697] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 4 LEADER]: Becoming Leader. State: Replica: 20ca690f4dc24bc2a051124def8a124d, State: Running, Role: LEADER
I20260709 13:10:03.097450 28325 consensus_queue.cc:237] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.098007 28033 raft_consensus.cc:2468] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ca690f4dc24bc2a051124def8a124d in term 4.
I20260709 13:10:03.108884 27930 catalog_manager.cc:5697] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d reported cstate change: term changed from 3 to 4, leader changed from 15aee576f44a4602894736f69680811b (127.27.63.66) to 20ca690f4dc24bc2a051124def8a124d (127.27.63.67). New cstate: current_term: 4 leader_uuid: "20ca690f4dc24bc2a051124def8a124d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:03.179104 28367 raft_consensus.cc:993] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b: : Instructing follower e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:10:03.179807 28366 raft_consensus.cc:1081] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Signalling peer e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:10:03.181851 28036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:56728
I20260709 13:10:03.182286 28036 raft_consensus.cc:493] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.182582 28036 raft_consensus.cc:3060] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.183014 28349 raft_consensus.cc:993] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b: : Instructing follower e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:10:03.183415 28366 raft_consensus.cc:1081] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Signalling peer e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:10:03.184899 28033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70e6679c989543a4a60743fbf5628180"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:56728
I20260709 13:10:03.185371 28033 raft_consensus.cc:493] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.185649 28033 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:03.187309 28036 raft_consensus.cc:515] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.188985 28036 leader_election.cc:290] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 2 election: Requested vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:10:03.189718 28110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:10:03.190099 28185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c1b8e756bf54b05b16b4d821d855df8" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:10:03.190361 28110 raft_consensus.cc:3055] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:03.190613 28185 raft_consensus.cc:3060] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.190727 28110 raft_consensus.cc:740] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:03.191354 28033 raft_consensus.cc:515] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.191674 28110 consensus_queue.cc:260] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.193076 28110 raft_consensus.cc:3060] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.194218 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:10:03.194607 28183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e6679c989543a4a60743fbf5628180" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:10:03.194751 28109 raft_consensus.cc:3055] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:03.195043 28109 raft_consensus.cc:740] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:03.195094 28183 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:03.195770 28109 consensus_queue.cc:260] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.196822 28109 raft_consensus.cc:3060] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:03.198066 28185 raft_consensus.cc:2468] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 2.
I20260709 13:10:03.193894 28033 leader_election.cc:290] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 3 election: Requested vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:10:03.200521 28000 leader_election.cc:304] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:10:03.200899 28110 raft_consensus.cc:2468] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 2.
I20260709 13:10:03.203097 28375 raft_consensus.cc:2804] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:03.203320 28183 raft_consensus.cc:2468] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 3.
I20260709 13:10:03.204473 28000 leader_election.cc:304] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:10:03.205292 28109 raft_consensus.cc:2468] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 3.
I20260709 13:10:03.206440 28375 raft_consensus.cc:697] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Becoming Leader. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:10:03.207522 28377 raft_consensus.cc:2804] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:03.207412 28375 consensus_queue.cc:237] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.208114 28377 raft_consensus.cc:697] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 LEADER]: Becoming Leader. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:10:03.208987 28377 consensus_queue.cc:237] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.215591 27930 catalog_manager.cc:5697] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 reported cstate change: term changed from 1 to 2, leader changed from 15aee576f44a4602894736f69680811b (127.27.63.66) to e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.226684 27930 catalog_manager.cc:5697] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 reported cstate change: term changed from 2 to 3, leader changed from 15aee576f44a4602894736f69680811b (127.27.63.66) to e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65). New cstate: current_term: 3 leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.244323 28349 raft_consensus.cc:993] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b: : Instructing follower e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:10:03.244776 28366 raft_consensus.cc:1081] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 4 LEADER]: Signalling peer e2890ef8c2cf47b7b0c8dbf5d27877a8 to start an election
I20260709 13:10:03.246523 28033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e"
dest_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8"
 from {username='slave'} at 127.0.0.1:56728
I20260709 13:10:03.247020 28033 raft_consensus.cc:493] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.247318 28033 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.252399 28033 raft_consensus.cc:515] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.254225 28033 leader_election.cc:290] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 5 election: Requested vote from peers 15aee576f44a4602894736f69680811b (127.27.63.66:40925), 20ca690f4dc24bc2a051124def8a124d (127.27.63.67:41679)
I20260709 13:10:03.255336 28109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "15aee576f44a4602894736f69680811b"
I20260709 13:10:03.255998 28109 raft_consensus.cc:3055] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:03.256323 28109 raft_consensus.cc:740] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 15aee576f44a4602894736f69680811b, State: Running, Role: LEADER
I20260709 13:10:03.255337 28183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92501d951008475fa44983bc16a8fb9e" candidate_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "20ca690f4dc24bc2a051124def8a124d"
I20260709 13:10:03.256906 28183 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.257045 28109 consensus_queue.cc:260] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.258188 28109 raft_consensus.cc:3060] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.263535 28183 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 5.
I20260709 13:10:03.264281 28109 raft_consensus.cc:2468] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2890ef8c2cf47b7b0c8dbf5d27877a8 in term 5.
I20260709 13:10:03.264686 28000 leader_election.cc:304] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d, e2890ef8c2cf47b7b0c8dbf5d27877a8; no voters: 
I20260709 13:10:03.265353 28377 raft_consensus.cc:2804] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:03.265750 28377 raft_consensus.cc:697] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 5 LEADER]: Becoming Leader. State: Replica: e2890ef8c2cf47b7b0c8dbf5d27877a8, State: Running, Role: LEADER
I20260709 13:10:03.266456 28377 consensus_queue.cc:237] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:03.273614 27930 catalog_manager.cc:5697] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 reported cstate change: term changed from 4 to 5, leader changed from 15aee576f44a4602894736f69680811b (127.27.63.66) to e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65). New cstate: current_term: 5 leader_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.477173 28033 raft_consensus.cc:1275] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Refusing update from remote peer 20ca690f4dc24bc2a051124def8a124d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:03.478543 28325 consensus_queue.cc:1048] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:03.490815 28109 raft_consensus.cc:1275] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Refusing update from remote peer 20ca690f4dc24bc2a051124def8a124d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:03.492179 28325 consensus_queue.cc:1048] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:03.566155 28033 raft_consensus.cc:1275] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Refusing update from remote peer 20ca690f4dc24bc2a051124def8a124d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:03.567408 28325 consensus_queue.cc:1048] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:03.577852 28109 raft_consensus.cc:1275] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Refusing update from remote peer 20ca690f4dc24bc2a051124def8a124d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:03.579321 28387 consensus_queue.cc:1048] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:03.614989 28183 raft_consensus.cc:1275] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:03.616688 28377 consensus_queue.cc:1048] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:03.625368 28109 raft_consensus.cc:1275] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:03.627097 28376 consensus_queue.cc:1048] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:03.681322 28109 raft_consensus.cc:1275] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:03.682503 28376 consensus_queue.cc:1048] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:03.691726 28183 raft_consensus.cc:1275] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:03.693095 28377 consensus_queue.cc:1048] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:10:03.723537 28132 debug-util.cc:398] Leaking SignalData structure 0x7b080012d080 after lost signal to thread 27904
W20260709 13:10:03.724462 28132 debug-util.cc:398] Leaking SignalData structure 0x7b080012d760 after lost signal to thread 27967
W20260709 13:10:03.725164 28132 debug-util.cc:398] Leaking SignalData structure 0x7b08001e2e80 after lost signal to thread 28060
W20260709 13:10:03.725768 28132 debug-util.cc:398] Leaking SignalData structure 0x7b08000a64a0 after lost signal to thread 28135
W20260709 13:10:03.726409 28132 debug-util.cc:398] Leaking SignalData structure 0x7b08001ea860 after lost signal to thread 28209
I20260709 13:10:03.804992 28109 raft_consensus.cc:1275] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 5 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:03.806222 28377 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:03.818251 28183 raft_consensus.cc:1275] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 5 FOLLOWER]: Refusing update from remote peer e2890ef8c2cf47b7b0c8dbf5d27877a8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:03.819911 28376 consensus_queue.cc:1048] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:03.966204 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:03.971920 27901 tablet_server.cc:179] TabletServer@127.27.63.65:0 shutting down...
I20260709 13:10:04.005789 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:04.006585 27901 tablet_replica.cc:333] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.007413 27901 raft_consensus.cc:2243] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.007906 27901 raft_consensus.cc:2272] T 787dd4bb42b548c39e14ada836c30081 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.010890 27901 tablet_replica.cc:333] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.011592 27901 raft_consensus.cc:2243] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:04.012568 27901 raft_consensus.cc:2272] T 0c1b8e756bf54b05b16b4d821d855df8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.014801 27901 tablet_replica.cc:333] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.015460 27901 raft_consensus.cc:2243] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.016023 27901 raft_consensus.cc:2272] T 1e5120e7f37845eb977a87e7eca3b017 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.018442 27901 tablet_replica.cc:333] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.019170 27901 raft_consensus.cc:2243] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.019732 27901 raft_consensus.cc:2272] T be4ae96b1b064518844fc56e9296677d P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.022212 27901 tablet_replica.cc:333] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.022866 27901 raft_consensus.cc:2243] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:04.023872 27901 raft_consensus.cc:2272] T 70e6679c989543a4a60743fbf5628180 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.025821 27901 tablet_replica.cc:333] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.026314 27901 raft_consensus.cc:2243] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:10:04.026989 27901 raft_consensus.cc:2272] T 92501d951008475fa44983bc16a8fb9e P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.028942 27901 tablet_replica.cc:333] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.029410 27901 raft_consensus.cc:2243] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.029794 27901 raft_consensus.cc:2272] T 6faee2f44c7e44db9f459461b9279789 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.031368 27901 tablet_replica.cc:333] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.031786 27901 raft_consensus.cc:2243] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.032183 27901 raft_consensus.cc:2272] T ce59f69f404a4584bce3d7f4c5cd8476 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.033763 27901 tablet_replica.cc:333] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8: stopping tablet replica
I20260709 13:10:04.034181 27901 raft_consensus.cc:2243] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.034575 27901 raft_consensus.cc:2272] T 5db7d5960e184238a5cecc537f2aedd8 P e2890ef8c2cf47b7b0c8dbf5d27877a8 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:04.199628 28150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111)
I20260709 13:10:04.203826 27901 tablet_server.cc:196] TabletServer@127.27.63.65:0 shutdown complete.
W20260709 13:10:04.209401 28150 consensus_peers.cc:597] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d -> Peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Couldn't send request to peer e2890ef8c2cf47b7b0c8dbf5d27877a8. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.219620 28076 consensus_peers.cc:597] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b -> Peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Couldn't send request to peer e2890ef8c2cf47b7b0c8dbf5d27877a8. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.235809 27901 tablet_server.cc:179] TabletServer@127.27.63.66:0 shutting down...
I20260709 13:10:04.271739 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:04.272737 27901 tablet_replica.cc:333] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.273646 27901 raft_consensus.cc:2243] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:04.274642 27901 raft_consensus.cc:2272] T 6faee2f44c7e44db9f459461b9279789 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.276515 27901 tablet_replica.cc:333] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.277017 27901 raft_consensus.cc:2243] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.277395 27901 raft_consensus.cc:2272] T 92501d951008475fa44983bc16a8fb9e P 15aee576f44a4602894736f69680811b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.279901 27901 tablet_replica.cc:333] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.280910 27901 raft_consensus.cc:2243] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.281474 27901 raft_consensus.cc:2272] T 70e6679c989543a4a60743fbf5628180 P 15aee576f44a4602894736f69680811b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.284088 27901 tablet_replica.cc:333] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.284622 27901 raft_consensus.cc:2243] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.285035 27901 raft_consensus.cc:2272] T 787dd4bb42b548c39e14ada836c30081 P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.287974 27901 tablet_replica.cc:333] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.288712 27901 raft_consensus.cc:2243] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:04.289633 27901 raft_consensus.cc:2272] T 1e5120e7f37845eb977a87e7eca3b017 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.291532 27901 tablet_replica.cc:333] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.292184 27901 raft_consensus.cc:2243] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:04.293226 27901 raft_consensus.cc:2272] T be4ae96b1b064518844fc56e9296677d P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.294883 27901 tablet_replica.cc:333] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.295323 27901 raft_consensus.cc:2243] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.295688 27901 raft_consensus.cc:2272] T 0c1b8e756bf54b05b16b4d821d855df8 P 15aee576f44a4602894736f69680811b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.297735 27901 tablet_replica.cc:333] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.298187 27901 raft_consensus.cc:2243] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.298554 27901 raft_consensus.cc:2272] T 5db7d5960e184238a5cecc537f2aedd8 P 15aee576f44a4602894736f69680811b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:04.301185 27901 tablet_replica.cc:333] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b: stopping tablet replica
I20260709 13:10:04.301676 27901 raft_consensus.cc:2243] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:04.302134 27901 raft_consensus.cc:2272] T ce59f69f404a4584bce3d7f4c5cd8476 P 15aee576f44a4602894736f69680811b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:04.353399 28150 consensus_peers.cc:597] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d -> Peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Couldn't send request to peer 15aee576f44a4602894736f69680811b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.359601 28150 consensus_peers.cc:597] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d -> Peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Couldn't send request to peer e2890ef8c2cf47b7b0c8dbf5d27877a8. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.402176 28150 consensus_peers.cc:597] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d -> Peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Couldn't send request to peer e2890ef8c2cf47b7b0c8dbf5d27877a8. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.409642 28150 consensus_peers.cc:597] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d -> Peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Couldn't send request to peer 15aee576f44a4602894736f69680811b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.772606 28150 consensus_peers.cc:597] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d -> Peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Couldn't send request to peer 15aee576f44a4602894736f69680811b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:05.197141 28399 raft_consensus.cc:493] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e2890ef8c2cf47b7b0c8dbf5d27877a8)
I20260709 13:10:05.197733 28399 raft_consensus.cc:515] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:05.199992 28399 leader_election.cc:290] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
W20260709 13:10:05.203821 28150 leader_election.cc:336] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111)
W20260709 13:10:05.204720 28150 leader_election.cc:336] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111)
I20260709 13:10:05.205027 28150 leader_election.cc:304] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d; no voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:05.205566 28399 raft_consensus.cc:2749] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:05.324254 28399 raft_consensus.cc:493] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e2890ef8c2cf47b7b0c8dbf5d27877a8)
I20260709 13:10:05.324740 28399 raft_consensus.cc:515] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:05.326467 28399 leader_election.cc:290] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
W20260709 13:10:05.328836 27901 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 28132)
W20260709 13:10:05.330600 28150 leader_election.cc:336] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111)
W20260709 13:10:05.331120 28150 leader_election.cc:336] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111)
I20260709 13:10:05.331398 28150 leader_election.cc:304] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d; no voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:05.331930 28399 raft_consensus.cc:2749] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:05.421721 28399 raft_consensus.cc:493] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 15aee576f44a4602894736f69680811b)
I20260709 13:10:05.422159 28399 raft_consensus.cc:515] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:05.423730 28399 leader_election.cc:290] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
W20260709 13:10:05.427459 28150 leader_election.cc:336] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111)
W20260709 13:10:05.428162 28150 leader_election.cc:336] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111)
I20260709 13:10:05.428545 28150 leader_election.cc:304] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d; no voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:05.429255 28399 raft_consensus.cc:2749] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:05.506007 28399 raft_consensus.cc:493] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e2890ef8c2cf47b7b0c8dbf5d27877a8)
I20260709 13:10:05.506480 28399 raft_consensus.cc:515] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:05.508145 28399 leader_election.cc:290] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
W20260709 13:10:05.512097 28150 leader_election.cc:336] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111)
W20260709 13:10:05.512833 28150 leader_election.cc:336] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111)
I20260709 13:10:05.513244 28150 leader_election.cc:304] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d; no voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:05.513901 28399 raft_consensus.cc:2749] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:05.761124 28399 raft_consensus.cc:493] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 15aee576f44a4602894736f69680811b)
I20260709 13:10:05.761686 28399 raft_consensus.cc:515] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } }
I20260709 13:10:05.763945 28399 leader_election.cc:290] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
W20260709 13:10:05.767561 28150 leader_election.cc:336] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111)
W20260709 13:10:05.768656 28150 leader_election.cc:336] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111)
I20260709 13:10:05.768947 28150 leader_election.cc:304] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d; no voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:05.769490 28399 raft_consensus.cc:2749] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:05.792598 28399 raft_consensus.cc:493] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 15aee576f44a4602894736f69680811b)
I20260709 13:10:05.793066 28399 raft_consensus.cc:515] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2890ef8c2cf47b7b0c8dbf5d27877a8" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 37339 } } peers { permanent_uuid: "20ca690f4dc24bc2a051124def8a124d" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 41679 } } peers { permanent_uuid: "15aee576f44a4602894736f69680811b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 40925 } }
I20260709 13:10:05.794746 28399 leader_election.cc:290] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339), 15aee576f44a4602894736f69680811b (127.27.63.66:40925)
W20260709 13:10:05.798494 28150 leader_election.cc:336] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e2890ef8c2cf47b7b0c8dbf5d27877a8 (127.27.63.65:37339): Network error: Client connection negotiation failed: client connection to 127.27.63.65:37339: connect: Connection refused (error 111)
W20260709 13:10:05.799265 28150 leader_election.cc:336] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 15aee576f44a4602894736f69680811b (127.27.63.66:40925): Network error: Client connection negotiation failed: client connection to 127.27.63.66:40925: connect: Connection refused (error 111)
I20260709 13:10:05.799587 28150 leader_election.cc:304] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20ca690f4dc24bc2a051124def8a124d; no voters: 15aee576f44a4602894736f69680811b, e2890ef8c2cf47b7b0c8dbf5d27877a8
I20260709 13:10:05.800141 28399 raft_consensus.cc:2749] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:06.003665 27901 tablet_server.cc:196] TabletServer@127.27.63.66:0 shutdown complete.
I20260709 13:10:06.022679 27901 tablet_server.cc:179] TabletServer@127.27.63.67:0 shutting down...
I20260709 13:10:06.047147 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:06.047837 27901 tablet_replica.cc:333] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.048447 27901 raft_consensus.cc:2243] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:06.048899 27901 raft_consensus.cc:2272] T 0c1b8e756bf54b05b16b4d821d855df8 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.050467 27901 tablet_replica.cc:333] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.051004 27901 raft_consensus.cc:2243] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:06.051412 27901 raft_consensus.cc:2272] T 70e6679c989543a4a60743fbf5628180 P 20ca690f4dc24bc2a051124def8a124d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.053026 27901 tablet_replica.cc:333] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.053472 27901 raft_consensus.cc:2243] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:06.053869 27901 raft_consensus.cc:2272] T 1e5120e7f37845eb977a87e7eca3b017 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.055311 27901 tablet_replica.cc:333] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.055779 27901 raft_consensus.cc:2243] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:06.056192 27901 raft_consensus.cc:2272] T be4ae96b1b064518844fc56e9296677d P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.057775 27901 tablet_replica.cc:333] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.058215 27901 raft_consensus.cc:2243] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:06.058862 27901 raft_consensus.cc:2272] T 787dd4bb42b548c39e14ada836c30081 P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.060469 27901 tablet_replica.cc:333] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.060926 27901 raft_consensus.cc:2243] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:06.061309 27901 raft_consensus.cc:2272] T 92501d951008475fa44983bc16a8fb9e P 20ca690f4dc24bc2a051124def8a124d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.062718 27901 tablet_replica.cc:333] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.063165 27901 raft_consensus.cc:2243] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:06.063551 27901 raft_consensus.cc:2272] T 6faee2f44c7e44db9f459461b9279789 P 20ca690f4dc24bc2a051124def8a124d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.065127 27901 tablet_replica.cc:333] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.065564 27901 raft_consensus.cc:2243] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:06.066213 27901 raft_consensus.cc:2272] T ce59f69f404a4584bce3d7f4c5cd8476 P 20ca690f4dc24bc2a051124def8a124d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.067683 27901 tablet_replica.cc:333] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d: stopping tablet replica
I20260709 13:10:06.068146 27901 raft_consensus.cc:2243] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:06.068774 27901 raft_consensus.cc:2272] T 5db7d5960e184238a5cecc537f2aedd8 P 20ca690f4dc24bc2a051124def8a124d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.095970 27901 tablet_server.cc:196] TabletServer@127.27.63.67:0 shutdown complete.
I20260709 13:10:06.114300 27901 master.cc:562] Master@127.27.63.126:44211 shutting down...
I20260709 13:10:06.242918 27901 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:06.243530 27901 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:06.243912 27901 tablet_replica.cc:333] T 00000000000000000000000000000000 P 28d419f2cdcb4937aa84c5b9d69f1a95: stopping tablet replica
I20260709 13:10:06.264007 27901 master.cc:584] Master@127.27.63.126:44211 shutdown complete.
15aee576f44a4602894736f69680811b  4
20ca690f4dc24bc2a051124def8a124d  1
e2890ef8c2cf47b7b0c8dbf5d27877a8  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (14529 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:10:06.308622 27901 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.63.126:35637
I20260709 13:10:06.309707 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:06.315177 28403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:06.315286 28402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:06.316005 28405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:06.317814 27901 server_base.cc:1061] running on GCE node
I20260709 13:10:06.318598 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:06.318804 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:06.318961 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602606318943 us; error 0 us; skew 500 ppm
I20260709 13:10:06.319514 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:06.321918 27901 webserver.cc:533] Webserver started at http://127.27.63.126:43923/ using document root <none> and password file <none>
I20260709 13:10:06.322432 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:06.322635 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:06.322906 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:06.324007 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/master-0-root/instance:
uuid: "67a71f07f30e4694b2e49d9516f84df7"
format_stamp: "Formatted at 2026-07-09 13:10:06 on dist-test-slave-sjgx"
I20260709 13:10:06.328740 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:10:06.331898 28410 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.332825 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:10:06.333070 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/master-0-root
uuid: "67a71f07f30e4694b2e49d9516f84df7"
format_stamp: "Formatted at 2026-07-09 13:10:06 on dist-test-slave-sjgx"
I20260709 13:10:06.333305 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:06.355072 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:06.356295 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:06.398404 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.126:35637
I20260709 13:10:06.398490 28461 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.126:35637 every 8 connection(s)
I20260709 13:10:06.402400 28462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.411873 28462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7: Bootstrap starting.
I20260709 13:10:06.416589 28462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.421059 28462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7: No bootstrap required, opened a new log
I20260709 13:10:06.423477 28462 raft_consensus.cc:359] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67a71f07f30e4694b2e49d9516f84df7" member_type: VOTER }
I20260709 13:10:06.423861 28462 raft_consensus.cc:385] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.424182 28462 raft_consensus.cc:740] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a71f07f30e4694b2e49d9516f84df7, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.424747 28462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67a71f07f30e4694b2e49d9516f84df7" member_type: VOTER }
I20260709 13:10:06.425185 28462 raft_consensus.cc:399] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:06.425412 28462 raft_consensus.cc:493] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:06.425681 28462 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:06.429693 28462 raft_consensus.cc:515] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67a71f07f30e4694b2e49d9516f84df7" member_type: VOTER }
I20260709 13:10:06.430192 28462 leader_election.cc:304] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67a71f07f30e4694b2e49d9516f84df7; no voters: 
I20260709 13:10:06.431296 28462 leader_election.cc:290] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:06.431720 28465 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:06.433058 28465 raft_consensus.cc:697] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 1 LEADER]: Becoming Leader. State: Replica: 67a71f07f30e4694b2e49d9516f84df7, State: Running, Role: LEADER
I20260709 13:10:06.433753 28465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67a71f07f30e4694b2e49d9516f84df7" member_type: VOTER }
I20260709 13:10:06.434172 28462 sys_catalog.cc:565] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:06.438334 28466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67a71f07f30e4694b2e49d9516f84df7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67a71f07f30e4694b2e49d9516f84df7" member_type: VOTER } }
I20260709 13:10:06.438385 28467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67a71f07f30e4694b2e49d9516f84df7. Latest consensus state: current_term: 1 leader_uuid: "67a71f07f30e4694b2e49d9516f84df7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67a71f07f30e4694b2e49d9516f84df7" member_type: VOTER } }
I20260709 13:10:06.439321 28466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:06.439376 28467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:06.442276 28472 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:06.447173 28472 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:06.450870 27901 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:06.456458 28472 catalog_manager.cc:1383] Generated new cluster ID: eea8a28b87794b3bb8825a418cc9063c
I20260709 13:10:06.456780 28472 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:06.479799 28472 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:06.481045 28472 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:06.501655 28472 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7: Generated new TSK 0
I20260709 13:10:06.502265 28472 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:06.518179 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:06.524739 28483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:06.525295 28484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:06.526803 28486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:06.527267 27901 server_base.cc:1061] running on GCE node
I20260709 13:10:06.528337 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:06.528585 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:06.528750 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602606528731 us; error 0 us; skew 500 ppm
I20260709 13:10:06.529256 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:06.531431 27901 webserver.cc:533] Webserver started at http://127.27.63.65:43211/ using document root <none> and password file <none>
I20260709 13:10:06.531903 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:06.532135 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:06.532405 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:06.533413 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-0-root/instance:
uuid: "43996ee009d24ce39b66709797501e9d"
format_stamp: "Formatted at 2026-07-09 13:10:06 on dist-test-slave-sjgx"
I20260709 13:10:06.538257 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:06.541893 28491 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.542932 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:10:06.543210 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "43996ee009d24ce39b66709797501e9d"
format_stamp: "Formatted at 2026-07-09 13:10:06 on dist-test-slave-sjgx"
I20260709 13:10:06.543490 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:06.554404 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:06.555619 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:06.557291 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:06.559541 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:06.559727 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.560007 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:06.560228 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:10:06.606231 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:39821
I20260709 13:10:06.606353 28553 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:39821 every 8 connection(s)
I20260709 13:10:06.612198 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:06.619019 28558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:06.621126 28559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:06.628409 27901 server_base.cc:1061] running on GCE node
W20260709 13:10:06.628537 28561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:06.628819 28554 heartbeater.cc:344] Connected to a master server at 127.27.63.126:35637
I20260709 13:10:06.629266 28554 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:06.630105 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:10:06.630177 28554 heartbeater.cc:507] Master 127.27.63.126:35637 requested a full tablet report, sending...
W20260709 13:10:06.630440 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:06.630654 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602606630634 us; error 0 us; skew 500 ppm
I20260709 13:10:06.631315 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:06.632648 28427 ts_manager.cc:194] Registered new tserver with Master: 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821)
I20260709 13:10:06.634066 27901 webserver.cc:533] Webserver started at http://127.27.63.66:38429/ using document root <none> and password file <none>
I20260709 13:10:06.634728 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:06.634971 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:06.635105 28427 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60248
I20260709 13:10:06.635306 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:06.636794 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-1-root/instance:
uuid: "cb418d9a7051481d976d42c15af78d0c"
format_stamp: "Formatted at 2026-07-09 13:10:06 on dist-test-slave-sjgx"
I20260709 13:10:06.641513 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:10:06.646309 28566 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.647294 27901 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:10:06.647614 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-1-root
uuid: "cb418d9a7051481d976d42c15af78d0c"
format_stamp: "Formatted at 2026-07-09 13:10:06 on dist-test-slave-sjgx"
I20260709 13:10:06.647940 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:06.659631 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:06.660898 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:06.662395 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:06.664623 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:06.664840 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.665074 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:06.665239 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.705961 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.66:36721
I20260709 13:10:06.706092 28628 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.66:36721 every 8 connection(s)
I20260709 13:10:06.710680 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:06.717624 28632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:06.719561 28633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:06.723187 28629 heartbeater.cc:344] Connected to a master server at 127.27.63.126:35637
I20260709 13:10:06.723738 28629 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:06.724895 28629 heartbeater.cc:507] Master 127.27.63.126:35637 requested a full tablet report, sending...
W20260709 13:10:06.725263 28635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:06.726169 27901 server_base.cc:1061] running on GCE node
I20260709 13:10:06.727016 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:06.727253 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:06.727155 28427 ts_manager.cc:194] Registered new tserver with Master: cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:06.727555 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602606727533 us; error 0 us; skew 500 ppm
I20260709 13:10:06.728184 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:06.728842 28427 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60258
I20260709 13:10:06.730701 27901 webserver.cc:533] Webserver started at http://127.27.63.67:40669/ using document root <none> and password file <none>
I20260709 13:10:06.731341 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:06.731571 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:06.731813 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:06.732936 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-2-root/instance:
uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
format_stamp: "Formatted at 2026-07-09 13:10:06 on dist-test-slave-sjgx"
I20260709 13:10:06.737304 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:10:06.740623 28640 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.741501 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:06.741887 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-2-root
uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
format_stamp: "Formatted at 2026-07-09 13:10:06 on dist-test-slave-sjgx"
I20260709 13:10:06.742197 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602591602838-27901-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:06.758395 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:06.759909 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:06.761732 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:06.764084 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:06.764319 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.764580 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:06.764755 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:06.804581 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.67:38283
I20260709 13:10:06.804666 28702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.67:38283 every 8 connection(s)
I20260709 13:10:06.818117 28703 heartbeater.cc:344] Connected to a master server at 127.27.63.126:35637
I20260709 13:10:06.818557 28703 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:06.819299 28703 heartbeater.cc:507] Master 127.27.63.126:35637 requested a full tablet report, sending...
I20260709 13:10:06.821247 28427 ts_manager.cc:194] Registered new tserver with Master: f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:06.822320 27901 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014548838s
I20260709 13:10:06.822724 28427 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60262
I20260709 13:10:06.851569 28427 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60268:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:06.854512 28427 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:06.925271 28594 tablet_service.cc:1511] Processing CreateTablet for tablet 20ddb6acaeec4670a9db1674759988df (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:06.926891 28594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ddb6acaeec4670a9db1674759988df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.927661 28593 tablet_service.cc:1511] Processing CreateTablet for tablet 171aa8eb9fb74f928c1abcf41b8a72ff (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:06.929137 28593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171aa8eb9fb74f928c1abcf41b8a72ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.928938 28591 tablet_service.cc:1511] Processing CreateTablet for tablet 308591aaed574bc5a1e9391c834f01dc (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:06.930318 28591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308591aaed574bc5a1e9391c834f01dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.933089 28592 tablet_service.cc:1511] Processing CreateTablet for tablet c6dce59ce0034d3d8351861f2ff79b33 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:06.934470 28592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dce59ce0034d3d8351861f2ff79b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.937747 28590 tablet_service.cc:1511] Processing CreateTablet for tablet 31a71981d19c44d0be6a1d04a520c9c1 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:06.939286 28590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a71981d19c44d0be6a1d04a520c9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.962178 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 308591aaed574bc5a1e9391c834f01dc (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:06.962868 28519 tablet_service.cc:1511] Processing CreateTablet for tablet 20ddb6acaeec4670a9db1674759988df (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:06.963641 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308591aaed574bc5a1e9391c834f01dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.964259 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ddb6acaeec4670a9db1674759988df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.969166 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 171aa8eb9fb74f928c1abcf41b8a72ff (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:06.970641 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171aa8eb9fb74f928c1abcf41b8a72ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.970968 28718 tablet_bootstrap.cc:492] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:06.973253 28517 tablet_service.cc:1511] Processing CreateTablet for tablet c6dce59ce0034d3d8351861f2ff79b33 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:06.974293 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dce59ce0034d3d8351861f2ff79b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.977216 28515 tablet_service.cc:1511] Processing CreateTablet for tablet 31a71981d19c44d0be6a1d04a520c9c1 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:06.978580 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a71981d19c44d0be6a1d04a520c9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.980865 28514 tablet_service.cc:1511] Processing CreateTablet for tablet 7a083523f80d4904917b750d12d78d49 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:06.982309 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a083523f80d4904917b750d12d78d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.985150 28718 tablet_bootstrap.cc:654] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.985088 28668 tablet_service.cc:1511] Processing CreateTablet for tablet 20ddb6acaeec4670a9db1674759988df (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:06.986256 28668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ddb6acaeec4670a9db1674759988df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.987910 28719 tablet_bootstrap.cc:492] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:06.989287 28667 tablet_service.cc:1511] Processing CreateTablet for tablet 171aa8eb9fb74f928c1abcf41b8a72ff (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:06.990017 28512 tablet_service.cc:1511] Processing CreateTablet for tablet 20c3599ec7274ec59d965dac9cffcacb (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:06.990880 28667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171aa8eb9fb74f928c1abcf41b8a72ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.991544 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20c3599ec7274ec59d965dac9cffcacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.993405 28664 tablet_service.cc:1511] Processing CreateTablet for tablet 31a71981d19c44d0be6a1d04a520c9c1 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:06.994652 28719 tablet_bootstrap.cc:654] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.994704 28511 tablet_service.cc:1511] Processing CreateTablet for tablet 2ea28f8b3c534695b1888ed36e8fe6ac (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:06.996308 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ea28f8b3c534695b1888ed36e8fe6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.996228 28665 tablet_service.cc:1511] Processing CreateTablet for tablet 308591aaed574bc5a1e9391c834f01dc (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:06.998307 28718 tablet_bootstrap.cc:492] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:06.998762 28666 tablet_service.cc:1511] Processing CreateTablet for tablet c6dce59ce0034d3d8351861f2ff79b33 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:06.998778 28718 ts_tablet_manager.cc:1403] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.004s
I20260709 13:10:06.999739 28510 tablet_service.cc:1511] Processing CreateTablet for tablet 6843ff6a43264ddab2199aaa87feb79b (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:07.001122 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6843ff6a43264ddab2199aaa87feb79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.004122 28513 tablet_service.cc:1511] Processing CreateTablet for tablet 59d1bb38c07c45efa9d1694302f4853e (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:07.005649 28513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d1bb38c07c45efa9d1694302f4853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.006947 28718 raft_consensus.cc:359] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.007817 28718 raft_consensus.cc:385] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.012790 28590 tablet_service.cc:1511] Processing CreateTablet for tablet 59d1bb38c07c45efa9d1694302f4853e (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:07.014220 28590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d1bb38c07c45efa9d1694302f4853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.016394 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 7a083523f80d4904917b750d12d78d49 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:07.017036 28594 tablet_service.cc:1511] Processing CreateTablet for tablet 20c3599ec7274ec59d965dac9cffcacb (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:07.018409 28594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20c3599ec7274ec59d965dac9cffcacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.018988 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 59d1bb38c07c45efa9d1694302f4853e (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:07.021248 28593 tablet_service.cc:1511] Processing CreateTablet for tablet 6843ff6a43264ddab2199aaa87feb79b (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:07.022749 28593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6843ff6a43264ddab2199aaa87feb79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.023319 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 2ea28f8b3c534695b1888ed36e8fe6ac (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:07.021348 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 20c3599ec7274ec59d965dac9cffcacb (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:07.027309 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 6843ff6a43264ddab2199aaa87feb79b (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:07.029490 28591 tablet_service.cc:1511] Processing CreateTablet for tablet 2ea28f8b3c534695b1888ed36e8fe6ac (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:07.029372 28592 tablet_service.cc:1511] Processing CreateTablet for tablet 7a083523f80d4904917b750d12d78d49 (DEFAULT_TABLE table=test-workload [id=85785e9d537944d88bca7b3dd0f0a139]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:06.994943 28664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a71981d19c44d0be6a1d04a520c9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.008319 28718 raft_consensus.cc:740] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.043128 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ea28f8b3c534695b1888ed36e8fe6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.043574 28718 consensus_queue.cc:260] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.049525 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d1bb38c07c45efa9d1694302f4853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.052474 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a083523f80d4904917b750d12d78d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.056500 28665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308591aaed574bc5a1e9391c834f01dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.060439 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20c3599ec7274ec59d965dac9cffcacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.041221 28592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a083523f80d4904917b750d12d78d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.068449 28666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dce59ce0034d3d8351861f2ff79b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.078027 28591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ea28f8b3c534695b1888ed36e8fe6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.078050 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6843ff6a43264ddab2199aaa87feb79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:07.092532 28723 tablet_bootstrap.cc:492] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.094322 28719 tablet_bootstrap.cc:492] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.094858 28719 ts_tablet_manager.cc:1403] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.107s	user 0.003s	sys 0.032s
I20260709 13:10:07.099022 28723 tablet_bootstrap.cc:654] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.119021 28723 tablet_bootstrap.cc:492] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.119504 28723 ts_tablet_manager.cc:1403] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.007s
I20260709 13:10:07.121263 28718 ts_tablet_manager.cc:1434] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.117s	user 0.011s	sys 0.000s
I20260709 13:10:07.122865 28629 heartbeater.cc:499] Master 127.27.63.126:35637 was elected leader, sending a full tablet report...
I20260709 13:10:07.123317 28718 tablet_bootstrap.cc:492] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.129803 28719 raft_consensus.cc:359] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.130568 28719 raft_consensus.cc:385] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.130860 28719 raft_consensus.cc:740] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.126973 28723 raft_consensus.cc:359] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.136508 28723 raft_consensus.cc:385] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.136849 28723 raft_consensus.cc:740] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.137630 28723 consensus_queue.cc:260] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.151741 28703 heartbeater.cc:499] Master 127.27.63.126:35637 was elected leader, sending a full tablet report...
I20260709 13:10:07.151633 28719 consensus_queue.cc:260] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.154197 28718 tablet_bootstrap.cc:654] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.158331 28723 ts_tablet_manager.cc:1434] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.039s	user 0.015s	sys 0.001s
I20260709 13:10:07.159623 28723 tablet_bootstrap.cc:492] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.166486 28723 tablet_bootstrap.cc:654] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.167855 28554 heartbeater.cc:499] Master 127.27.63.126:35637 was elected leader, sending a full tablet report...
I20260709 13:10:07.168808 28719 ts_tablet_manager.cc:1434] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.074s	user 0.016s	sys 0.003s
I20260709 13:10:07.169526 28719 tablet_bootstrap.cc:492] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.171597 28718 tablet_bootstrap.cc:492] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.171941 28718 ts_tablet_manager.cc:1403] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.003s
I20260709 13:10:07.172021 28723 tablet_bootstrap.cc:492] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.172504 28723 ts_tablet_manager.cc:1403] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 13:10:07.175125 28718 raft_consensus.cc:359] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.175920 28718 raft_consensus.cc:385] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.175707 28723 raft_consensus.cc:359] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.176302 28718 raft_consensus.cc:740] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.176508 28723 raft_consensus.cc:385] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.176844 28723 raft_consensus.cc:740] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.177219 28718 consensus_queue.cc:260] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.177932 28719 tablet_bootstrap.cc:654] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.177613 28723 consensus_queue.cc:260] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.179809 28723 ts_tablet_manager.cc:1434] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:07.180847 28723 tablet_bootstrap.cc:492] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.184497 28718 ts_tablet_manager.cc:1434] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:10:07.185832 28718 tablet_bootstrap.cc:492] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.187539 28723 tablet_bootstrap.cc:654] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.200861 28723 tablet_bootstrap.cc:492] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.201409 28723 ts_tablet_manager.cc:1403] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 13:10:07.204555 28723 raft_consensus.cc:359] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.205377 28723 raft_consensus.cc:385] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.205711 28723 raft_consensus.cc:740] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.206678 28718 tablet_bootstrap.cc:654] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.206460 28723 consensus_queue.cc:260] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.207286 28719 tablet_bootstrap.cc:492] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.207762 28719 ts_tablet_manager.cc:1403] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.023s
I20260709 13:10:07.208807 28723 ts_tablet_manager.cc:1434] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:07.209823 28723 tablet_bootstrap.cc:492] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.210628 28719 raft_consensus.cc:359] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.211335 28719 raft_consensus.cc:385] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.211613 28719 raft_consensus.cc:740] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.212561 28719 consensus_queue.cc:260] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.214784 28719 ts_tablet_manager.cc:1434] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:07.215662 28719 tablet_bootstrap.cc:492] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.216768 28723 tablet_bootstrap.cc:654] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.222930 28719 tablet_bootstrap.cc:654] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.234757 28723 tablet_bootstrap.cc:492] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.235227 28723 ts_tablet_manager.cc:1403] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.003s
I20260709 13:10:07.235749 28718 tablet_bootstrap.cc:492] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.236231 28718 ts_tablet_manager.cc:1403] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.016s
I20260709 13:10:07.237583 28719 tablet_bootstrap.cc:492] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.237566 28723 raft_consensus.cc:359] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.238155 28719 ts_tablet_manager.cc:1403] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 13:10:07.238324 28723 raft_consensus.cc:385] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.238663 28723 raft_consensus.cc:740] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.239012 28718 raft_consensus.cc:359] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.239758 28718 raft_consensus.cc:385] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.239323 28723 consensus_queue.cc:260] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.240131 28718 raft_consensus.cc:740] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.240939 28718 consensus_queue.cc:260] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.241184 28719 raft_consensus.cc:359] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.242054 28723 ts_tablet_manager.cc:1434] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:10:07.243006 28723 tablet_bootstrap.cc:492] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.243067 28718 ts_tablet_manager.cc:1434] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:07.244115 28718 tablet_bootstrap.cc:492] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.245349 28725 raft_consensus.cc:493] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.245813 28725 raft_consensus.cc:515] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.249908 28723 tablet_bootstrap.cc:654] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.250465 28718 tablet_bootstrap.cc:654] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.262388 28725 leader_election.cc:290] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:07.268514 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
W20260709 13:10:07.269966 28643 leader_election.cc:343] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.270987 28723 tablet_bootstrap.cc:492] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.271420 28723 ts_tablet_manager.cc:1403] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 13:10:07.273571 28719 raft_consensus.cc:385] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.273384 28604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.274029 28719 raft_consensus.cc:740] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.274260 28604 raft_consensus.cc:2468] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 0.
I20260709 13:10:07.274744 28723 raft_consensus.cc:359] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.274916 28719 consensus_queue.cc:260] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.275460 28718 tablet_bootstrap.cc:492] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.275555 28723 raft_consensus.cc:385] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.275529 28642 leader_election.cc:304] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 43996ee009d24ce39b66709797501e9d
I20260709 13:10:07.276167 28718 ts_tablet_manager.cc:1403] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.019s
I20260709 13:10:07.276132 28723 raft_consensus.cc:740] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.276919 28725 raft_consensus.cc:2804] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.277328 28725 raft_consensus.cc:493] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.277161 28723 consensus_queue.cc:260] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.277683 28725 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.279256 28718 raft_consensus.cc:359] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.279948 28719 ts_tablet_manager.cc:1434] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.041s	user 0.001s	sys 0.005s
I20260709 13:10:07.280095 28718 raft_consensus.cc:385] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.280531 28718 raft_consensus.cc:740] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.281021 28719 tablet_bootstrap.cc:492] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.281324 28718 consensus_queue.cc:260] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.283636 28718 ts_tablet_manager.cc:1434] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:07.284595 28718 tablet_bootstrap.cc:492] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.284914 28725 raft_consensus.cc:515] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.287782 28719 tablet_bootstrap.cc:654] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.288151 28725 leader_election.cc:290] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:07.290658 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:07.291258 28718 tablet_bootstrap.cc:654] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:07.293769 28643 leader_election.cc:343] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.297024 28604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:07.297652 28604 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.314244 28718 tablet_bootstrap.cc:492] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.315079 28718 ts_tablet_manager.cc:1403] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.004s
I20260709 13:10:07.317864 28604 raft_consensus.cc:2468] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 1.
I20260709 13:10:07.319088 28718 raft_consensus.cc:359] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.319674 28642 leader_election.cc:304] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 43996ee009d24ce39b66709797501e9d
I20260709 13:10:07.320233 28718 raft_consensus.cc:385] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.320704 28718 raft_consensus.cc:740] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.321375 28725 raft_consensus.cc:2804] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.321960 28725 raft_consensus.cc:697] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Becoming Leader. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:07.321708 28718 consensus_queue.cc:260] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.323068 28725 consensus_queue.cc:237] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.328620 28718 ts_tablet_manager.cc:1434] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.005s
I20260709 13:10:07.329867 28718 tablet_bootstrap.cc:492] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.341786 28718 tablet_bootstrap.cc:654] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.354897 28718 tablet_bootstrap.cc:492] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.355780 28718 ts_tablet_manager.cc:1403] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.005s
I20260709 13:10:07.355573 28426 catalog_manager.cc:5697] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.360005 28718 raft_consensus.cc:359] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.361012 28718 raft_consensus.cc:385] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.361404 28718 raft_consensus.cc:740] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.362782 28723 ts_tablet_manager.cc:1434] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.090s	user 0.037s	sys 0.036s
I20260709 13:10:07.362349 28718 consensus_queue.cc:260] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.363847 28723 tablet_bootstrap.cc:492] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.364843 28719 tablet_bootstrap.cc:492] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.365329 28719 ts_tablet_manager.cc:1403] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.085s	user 0.028s	sys 0.047s
I20260709 13:10:07.366323 28722 raft_consensus.cc:493] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.366789 28722 raft_consensus.cc:515] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.368495 28719 raft_consensus.cc:359] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.369223 28719 raft_consensus.cc:385] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.369489 28719 raft_consensus.cc:740] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.370267 28719 consensus_queue.cc:260] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.374639 28719 ts_tablet_manager.cc:1434] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:10:07.375063 28723 tablet_bootstrap.cc:654] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.375705 28719 tablet_bootstrap.cc:492] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.376533 28722 leader_election.cc:290] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.387357 28719 tablet_bootstrap.cc:654] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.388772 28723 tablet_bootstrap.cc:492] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.389482 28723 ts_tablet_manager.cc:1403] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.011s
I20260709 13:10:07.390473 28718 ts_tablet_manager.cc:1434] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.004s
I20260709 13:10:07.391525 28718 tablet_bootstrap.cc:492] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.392855 28723 raft_consensus.cc:359] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.393671 28723 raft_consensus.cc:385] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.394162 28723 raft_consensus.cc:740] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.395190 28723 consensus_queue.cc:260] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.398634 28718 tablet_bootstrap.cc:654] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.399701 28719 tablet_bootstrap.cc:492] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.404712 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
I20260709 13:10:07.405323 28719 ts_tablet_manager.cc:1403] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.003s
W20260709 13:10:07.406423 28569 leader_election.cc:343] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.404755 28723 ts_tablet_manager.cc:1434] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:10:07.408090 28723 tablet_bootstrap.cc:492] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.408536 28726 raft_consensus.cc:493] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.408974 28726 raft_consensus.cc:515] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.415369 28723 tablet_bootstrap.cc:654] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.419629 28726 leader_election.cc:290] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.422120 28718 tablet_bootstrap.cc:492] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.422845 28718 ts_tablet_manager.cc:1403] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.005s
I20260709 13:10:07.424556 28719 raft_consensus.cc:359] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.425552 28719 raft_consensus.cc:385] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.425963 28719 raft_consensus.cc:740] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.426344 28718 raft_consensus.cc:359] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.427466 28718 raft_consensus.cc:385] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.427966 28718 raft_consensus.cc:740] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.429050 28718 consensus_queue.cc:260] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.431828 28718 ts_tablet_manager.cc:1434] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:07.432830 28718 tablet_bootstrap.cc:492] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.426978 28719 consensus_queue.cc:260] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.434604 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
W20260709 13:10:07.436602 28567 leader_election.cc:343] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.437069 28567 leader_election.cc:304] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c; no voters: 43996ee009d24ce39b66709797501e9d, f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:07.438278 28722 raft_consensus.cc:2749] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:07.441743 28718 tablet_bootstrap.cc:654] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.441938 28726 raft_consensus.cc:493] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.442587 28726 raft_consensus.cc:515] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.444660 28726 leader_election.cc:290] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.436735 28719 ts_tablet_manager.cc:1434] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.002s
W20260709 13:10:07.447125 28475 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:07.447772 28719 tablet_bootstrap.cc:492] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.451879 28723 tablet_bootstrap.cc:492] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.452445 28723 ts_tablet_manager.cc:1403] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.003s
I20260709 13:10:07.453481 28718 tablet_bootstrap.cc:492] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.453980 28718 ts_tablet_manager.cc:1403] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.007s
I20260709 13:10:07.455531 28719 tablet_bootstrap.cc:654] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.455432 28723 raft_consensus.cc:359] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.457971 28723 raft_consensus.cc:385] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.458343 28723 raft_consensus.cc:740] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.456686 28718 raft_consensus.cc:359] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.459757 28718 raft_consensus.cc:385] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.460247 28718 raft_consensus.cc:740] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.461030 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6843ff6a43264ddab2199aaa87feb79b" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
I20260709 13:10:07.461210 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308591aaed574bc5a1e9391c834f01dc" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
I20260709 13:10:07.461089 28718 consensus_queue.cc:260] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.461802 28678 raft_consensus.cc:2468] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 0.
I20260709 13:10:07.461874 28677 raft_consensus.cc:2468] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 0.
I20260709 13:10:07.462905 28722 raft_consensus.cc:493] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.463361 28722 raft_consensus.cc:515] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.465370 28722 leader_election.cc:290] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.465903 28718 ts_tablet_manager.cc:1434] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:10:07.465801 28492 leader_election.cc:304] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:07.466949 28726 raft_consensus.cc:2804] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.467010 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20c3599ec7274ec59d965dac9cffcacb" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
I20260709 13:10:07.467465 28726 raft_consensus.cc:493] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.467350 28492 leader_election.cc:304] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:07.467691 28677 raft_consensus.cc:2468] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 0.
I20260709 13:10:07.466935 28718 tablet_bootstrap.cc:492] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.469197 28567 leader_election.cc:304] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:07.459775 28723 consensus_queue.cc:260] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.467926 28726 raft_consensus.cc:3060] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.471040 28722 raft_consensus.cc:2804] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.471396 28722 raft_consensus.cc:493] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.471715 28722 raft_consensus.cc:3060] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.473419 28723 ts_tablet_manager.cc:1434] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:10:07.475419 28723 tablet_bootstrap.cc:492] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.477958 28718 tablet_bootstrap.cc:654] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.480849 28722 raft_consensus.cc:515] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.490942 28737 raft_consensus.cc:493] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.491144 28723 tablet_bootstrap.cc:654] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.491466 28737 raft_consensus.cc:515] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.492807 28753 raft_consensus.cc:493] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.493423 28753 raft_consensus.cc:515] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.494796 28737 leader_election.cc:290] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:07.495682 28722 leader_election.cc:290] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.495939 28604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6843ff6a43264ddab2199aaa87feb79b" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.496166 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308591aaed574bc5a1e9391c834f01dc" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.496979 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20c3599ec7274ec59d965dac9cffcacb" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:07.497978 28603 raft_consensus.cc:2468] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 0.
I20260709 13:10:07.497948 28604 raft_consensus.cc:2468] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 0.
I20260709 13:10:07.498278 28677 raft_consensus.cc:3060] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.497206 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea28f8b3c534695b1888ed36e8fe6ac" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
W20260709 13:10:07.500204 28642 leader_election.cc:343] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.501587 28753 leader_election.cc:290] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283), 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821)
I20260709 13:10:07.502699 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
I20260709 13:10:07.503559 28678 raft_consensus.cc:2468] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 0.
I20260709 13:10:07.504387 28726 raft_consensus.cc:515] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.505194 28567 leader_election.cc:304] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:07.506201 28753 raft_consensus.cc:2804] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.507406 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6843ff6a43264ddab2199aaa87feb79b" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:07.508123 28602 raft_consensus.cc:3060] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.508371 28719 tablet_bootstrap.cc:492] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.508865 28719 ts_tablet_manager.cc:1403] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.007s
I20260709 13:10:07.508719 28677 raft_consensus.cc:2468] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 1.
I20260709 13:10:07.510394 28567 leader_election.cc:304] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:07.511713 28722 raft_consensus.cc:2804] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.512333 28722 raft_consensus.cc:697] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Becoming Leader. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:07.512429 28737 raft_consensus.cc:493] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.513120 28737 raft_consensus.cc:515] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.513463 28722 consensus_queue.cc:237] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.515322 28726 leader_election.cc:290] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
W20260709 13:10:07.520087 28492 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:38283, user_credentials={real_user=slave}} blocked reactor thread for 52811.8us
I20260709 13:10:07.515759 28737 leader_election.cc:290] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:07.527015 28719 raft_consensus.cc:359] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.529066 28719 raft_consensus.cc:385] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.529595 28719 raft_consensus.cc:740] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.516357 28753 raft_consensus.cc:493] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.530472 28753 raft_consensus.cc:3060] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.529062 28758 raft_consensus.cc:493] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.531589 28758 raft_consensus.cc:515] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.531818 28719 consensus_queue.cc:260] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.531030 28726 raft_consensus.cc:2804] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.532982 28726 raft_consensus.cc:493] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.533341 28726 raft_consensus.cc:3060] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.534327 28758 leader_election.cc:290] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.535691 28604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.537480 28604 raft_consensus.cc:2468] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 0.
I20260709 13:10:07.538070 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.538962 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6843ff6a43264ddab2199aaa87feb79b" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
W20260709 13:10:07.539418 28642 leader_election.cc:343] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.539592 28677 raft_consensus.cc:3060] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.542327 28493 leader_election.cc:304] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:07.543540 28755 raft_consensus.cc:2804] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.544137 28755 raft_consensus.cc:493] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.544607 28755 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.545086 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea28f8b3c534695b1888ed36e8fe6ac" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
I20260709 13:10:07.545739 28602 raft_consensus.cc:2468] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 1.
I20260709 13:10:07.548544 28493 leader_election.cc:304] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:07.552816 28755 raft_consensus.cc:515] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.550539 28719 ts_tablet_manager.cc:1434] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 13:10:07.552924 28723 tablet_bootstrap.cc:492] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.554267 28723 ts_tablet_manager.cc:1403] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.079s	user 0.010s	sys 0.010s
I20260709 13:10:07.555629 28755 leader_election.cc:290] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.555500 28527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20c3599ec7274ec59d965dac9cffcacb" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:07.552675 28528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
I20260709 13:10:07.556854 28527 raft_consensus.cc:3060] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.556880 28526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
I20260709 13:10:07.557399 28528 raft_consensus.cc:2468] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 0.
I20260709 13:10:07.554134 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20c3599ec7274ec59d965dac9cffcacb" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
I20260709 13:10:07.559494 28723 raft_consensus.cc:359] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
W20260709 13:10:07.561769 28569 leader_election.cc:343] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.560511 28723 raft_consensus.cc:385] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.562587 28723 raft_consensus.cc:740] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
W20260709 13:10:07.563398 28643 leader_election.cc:343] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.563939 28643 leader_election.cc:304] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2d3a16ae3d14637b4e6aafe033a3157; no voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c
I20260709 13:10:07.564527 28753 raft_consensus.cc:515] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.561972 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
W20260709 13:10:07.567065 28492 leader_election.cc:343] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.567282 28753 leader_election.cc:290] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 election: Requested vote from peers f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283), 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821)
I20260709 13:10:07.568007 28758 raft_consensus.cc:2804] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.567205 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:07.568838 28758 raft_consensus.cc:697] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:07.569674 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:07.569769 28758 consensus_queue.cc:237] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.570086 28426 catalog_manager.cc:5697] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c reported cstate change: term changed from 0 to 1, leader changed from <none> to cb418d9a7051481d976d42c15af78d0c (127.27.63.66). New cstate: current_term: 1 leader_uuid: "cb418d9a7051481d976d42c15af78d0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:07.570928 28569 leader_election.cc:343] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:07.571123 28492 leader_election.cc:343] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.572098 28723 consensus_queue.cc:260] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.572656 28737 raft_consensus.cc:2749] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:07.572945 28725 raft_consensus.cc:493] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.572793 28643 leader_election.cc:304] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, f2d3a16ae3d14637b4e6aafe033a3157; no voters: cb418d9a7051481d976d42c15af78d0c
I20260709 13:10:07.574434 28737 raft_consensus.cc:2804] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.574854 28737 raft_consensus.cc:493] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.575229 28737 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.573407 28725 raft_consensus.cc:515] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.576704 28718 tablet_bootstrap.cc:492] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.577174 28718 ts_tablet_manager.cc:1403] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.110s	user 0.009s	sys 0.021s
I20260709 13:10:07.577869 28725 leader_election.cc:290] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:07.578861 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:07.579010 28726 raft_consensus.cc:515] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.579461 28678 raft_consensus.cc:3060] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.580295 28718 raft_consensus.cc:359] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.580875 28718 raft_consensus.cc:385] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.581112 28718 raft_consensus.cc:740] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.581739 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.582345 28603 raft_consensus.cc:2468] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 0.
I20260709 13:10:07.582177 28718 consensus_queue.cc:260] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.582782 28604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308591aaed574bc5a1e9391c834f01dc" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:07.583067 28677 raft_consensus.cc:2468] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 1.
I20260709 13:10:07.583339 28604 raft_consensus.cc:3060] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.583582 28642 leader_election.cc:304] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:07.585330 28718 ts_tablet_manager.cc:1434] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:07.586472 28718 tablet_bootstrap.cc:492] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c: Bootstrap starting.
I20260709 13:10:07.585256 28725 raft_consensus.cc:2804] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.587128 28725 raft_consensus.cc:493] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.574326 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:07.587481 28527 raft_consensus.cc:2468] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 1.
I20260709 13:10:07.591272 28604 raft_consensus.cc:2468] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 1.
I20260709 13:10:07.591845 28725 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.590806 28737 raft_consensus.cc:515] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.594726 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
I20260709 13:10:07.597213 28737 leader_election.cc:290] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:07.597859 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:07.598465 28603 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.600121 28493 leader_election.cc:304] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:07.592981 28602 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.601904 28758 raft_consensus.cc:2804] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.602665 28758 raft_consensus.cc:697] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:07.603492 28758 consensus_queue.cc:237] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.605763 28528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:07.606385 28528 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.608196 28725 raft_consensus.cc:515] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.607474 28719 tablet_bootstrap.cc:492] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.615471 28723 ts_tablet_manager.cc:1434] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 13:10:07.610054 28678 raft_consensus.cc:2468] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 1.
I20260709 13:10:07.606321 28603 raft_consensus.cc:2468] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 1.
I20260709 13:10:07.617777 28567 leader_election.cc:304] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 43996ee009d24ce39b66709797501e9d
I20260709 13:10:07.618696 28723 tablet_bootstrap.cc:492] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.619243 28722 raft_consensus.cc:2804] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.619729 28722 raft_consensus.cc:697] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Becoming Leader. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:07.620537 28722 consensus_queue.cc:237] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.623186 28642 leader_election.cc:304] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:07.624372 28737 raft_consensus.cc:2804] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.624754 28723 tablet_bootstrap.cc:654] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.629448 28719 tablet_bootstrap.cc:654] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.629722 28737 raft_consensus.cc:697] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Becoming Leader. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:07.637451 28737 consensus_queue.cc:237] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.638394 28425 catalog_manager.cc:5697] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 1 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.631590 28602 raft_consensus.cc:2468] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 1.
I20260709 13:10:07.643036 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:07.644708 28493 leader_election.cc:304] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:07.644625 28427 catalog_manager.cc:5697] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c reported cstate change: term changed from 0 to 1, leader changed from <none> to cb418d9a7051481d976d42c15af78d0c (127.27.63.66). New cstate: current_term: 1 leader_uuid: "cb418d9a7051481d976d42c15af78d0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.615483 28528 raft_consensus.cc:2468] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 1.
I20260709 13:10:07.647926 28755 raft_consensus.cc:2804] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.650218 28755 raft_consensus.cc:697] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:07.650643 28767 raft_consensus.cc:493] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.617779 28718 tablet_bootstrap.cc:654] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.651111 28755 consensus_queue.cc:237] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.633733 28726 leader_election.cc:290] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
W20260709 13:10:07.634869 28643 leader_election.cc:343] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:07.656852 28643 leader_election.cc:343] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.635561 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:07.636195 28725 leader_election.cc:290] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:07.657631 28603 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.657871 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308591aaed574bc5a1e9391c834f01dc" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:07.658496 28678 raft_consensus.cc:3060] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.667249 28603 raft_consensus.cc:2468] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 1.
I20260709 13:10:07.668928 28642 leader_election.cc:304] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 43996ee009d24ce39b66709797501e9d
I20260709 13:10:07.670059 28678 raft_consensus.cc:2468] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 1.
I20260709 13:10:07.670277 28725 raft_consensus.cc:2804] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.675113 28725 raft_consensus.cc:697] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Becoming Leader. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:07.674407 28723 tablet_bootstrap.cc:492] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.676381 28723 ts_tablet_manager.cc:1403] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.006s
I20260709 13:10:07.676141 28725 consensus_queue.cc:237] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.677814 28427 catalog_manager.cc:5697] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 1 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.678797 28719 tablet_bootstrap.cc:492] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.679311 28719 ts_tablet_manager.cc:1403] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.087s	user 0.011s	sys 0.004s
I20260709 13:10:07.679407 28723 raft_consensus.cc:359] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.680343 28723 raft_consensus.cc:385] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.680757 28723 raft_consensus.cc:740] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.681638 28718 tablet_bootstrap.cc:492] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c: No bootstrap required, opened a new log
I20260709 13:10:07.682178 28718 ts_tablet_manager.cc:1403] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c: Time spent bootstrapping tablet: real 0.096s	user 0.011s	sys 0.000s
I20260709 13:10:07.681649 28723 consensus_queue.cc:260] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.682564 28719 raft_consensus.cc:359] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.683362 28719 raft_consensus.cc:385] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.683725 28719 raft_consensus.cc:740] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.684602 28723 ts_tablet_manager.cc:1434] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:10:07.684587 28719 consensus_queue.cc:260] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.684878 28718 raft_consensus.cc:359] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.651098 28767 raft_consensus.cc:515] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.687525 28767 leader_election.cc:290] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.688126 28718 raft_consensus.cc:385] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.688453 28719 ts_tablet_manager.cc:1434] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.006s
I20260709 13:10:07.688319 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.689327 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
I20260709 13:10:07.689779 28719 tablet_bootstrap.cc:492] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.689491 28603 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 0.
I20260709 13:10:07.689931 28678 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 0.
I20260709 13:10:07.691429 28493 leader_election.cc:304] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:07.693143 28723 tablet_bootstrap.cc:492] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157: Bootstrap starting.
I20260709 13:10:07.697023 28719 tablet_bootstrap.cc:654] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.697780 28767 raft_consensus.cc:2804] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.698206 28767 raft_consensus.cc:493] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.698586 28767 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.699255 28723 tablet_bootstrap.cc:654] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.688463 28718 raft_consensus.cc:740] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.700434 28718 consensus_queue.cc:260] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.702589 28718 ts_tablet_manager.cc:1434] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.001s
I20260709 13:10:07.709209 28425 catalog_manager.cc:5697] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 1 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.710819 28767 raft_consensus.cc:515] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.711077 28723 tablet_bootstrap.cc:492] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157: No bootstrap required, opened a new log
I20260709 13:10:07.711736 28723 ts_tablet_manager.cc:1403] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.008s
I20260709 13:10:07.713904 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:07.714569 28603 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.714435 28723 raft_consensus.cc:359] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.715106 28723 raft_consensus.cc:385] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.718973 28723 raft_consensus.cc:740] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.719798 28723 consensus_queue.cc:260] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.722215 28723 ts_tablet_manager.cc:1434] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:10:07.722489 28603 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 1.
I20260709 13:10:07.723752 28493 leader_election.cc:304] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:07.724705 28755 raft_consensus.cc:2804] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.725251 28719 tablet_bootstrap.cc:492] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.725231 28755 raft_consensus.cc:697] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:07.724846 28427 catalog_manager.cc:5697] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.725833 28719 ts_tablet_manager.cc:1403] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.005s
I20260709 13:10:07.726243 28755 consensus_queue.cc:237] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.726768 28427 catalog_manager.cc:5697] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.728848 28719 raft_consensus.cc:359] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.729580 28719 raft_consensus.cc:385] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.729929 28719 raft_consensus.cc:740] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.730902 28719 consensus_queue.cc:260] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.733566 28719 ts_tablet_manager.cc:1434] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:07.734617 28719 tablet_bootstrap.cc:492] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.736099 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:07.736107 28767 leader_election.cc:290] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.736760 28678 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.739573 28426 catalog_manager.cc:5697] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 1 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.741017 28719 tablet_bootstrap.cc:654] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.743428 28678 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 1.
I20260709 13:10:07.747650 28719 tablet_bootstrap.cc:492] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.748193 28719 ts_tablet_manager.cc:1403] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 13:10:07.750813 28719 raft_consensus.cc:359] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.751552 28719 raft_consensus.cc:385] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.751892 28719 raft_consensus.cc:740] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.751981 28722 raft_consensus.cc:493] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.752823 28719 consensus_queue.cc:260] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.752629 28722 raft_consensus.cc:515] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.754287 28719 ts_tablet_manager.cc:1434] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:07.755148 28719 tablet_bootstrap.cc:492] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d: Bootstrap starting.
I20260709 13:10:07.756238 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea28f8b3c534695b1888ed36e8fe6ac" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
I20260709 13:10:07.756197 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea28f8b3c534695b1888ed36e8fe6ac" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
I20260709 13:10:07.756846 28529 raft_consensus.cc:2468] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 0.
I20260709 13:10:07.756933 28678 raft_consensus.cc:2468] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 0.
I20260709 13:10:07.757992 28569 leader_election.cc:304] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:07.758975 28753 raft_consensus.cc:2804] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.759009 28722 leader_election.cc:290] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.759500 28753 raft_consensus.cc:493] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.759892 28753 raft_consensus.cc:3060] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.760462 28719 tablet_bootstrap.cc:654] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.764765 28719 tablet_bootstrap.cc:492] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d: No bootstrap required, opened a new log
I20260709 13:10:07.765193 28719 ts_tablet_manager.cc:1403] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d: Time spent bootstrapping tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:10:07.765748 28753 raft_consensus.cc:515] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.767376 28719 raft_consensus.cc:359] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.768102 28719 raft_consensus.cc:385] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.768327 28719 raft_consensus.cc:740] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.768827 28719 consensus_queue.cc:260] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.769135 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea28f8b3c534695b1888ed36e8fe6ac" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:07.769767 28529 raft_consensus.cc:3060] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.770473 28719 ts_tablet_manager.cc:1434] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:07.771467 28753 leader_election.cc:290] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.772899 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea28f8b3c534695b1888ed36e8fe6ac" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:07.773460 28678 raft_consensus.cc:3060] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.777062 28529 raft_consensus.cc:2468] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 1.
I20260709 13:10:07.778329 28528 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.778607 28569 leader_election.cc:304] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:07.779054 28678 raft_consensus.cc:2468] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 1.
I20260709 13:10:07.779477 28753 raft_consensus.cc:2804] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.780699 28753 raft_consensus.cc:697] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Becoming Leader. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:07.781634 28753 consensus_queue.cc:237] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.786650 28725 consensus_queue.cc:1048] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.788156 28767 raft_consensus.cc:493] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.788633 28767 raft_consensus.cc:515] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:07.790747 28767 leader_election.cc:290] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.790853 28426 catalog_manager.cc:5697] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c reported cstate change: term changed from 0 to 1, leader changed from <none> to cb418d9a7051481d976d42c15af78d0c (127.27.63.66). New cstate: current_term: 1 leader_uuid: "cb418d9a7051481d976d42c15af78d0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.791991 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
I20260709 13:10:07.792582 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.792840 28678 raft_consensus.cc:2393] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43996ee009d24ce39b66709797501e9d in current term 1: Already voted for candidate cb418d9a7051481d976d42c15af78d0c in this term.
I20260709 13:10:07.795348 28492 leader_election.cc:304] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43996ee009d24ce39b66709797501e9d; no voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:07.796223 28767 raft_consensus.cc:3060] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.802568 28767 raft_consensus.cc:2749] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:07.809108 28725 consensus_queue.cc:1048] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.862350 28603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df"
dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
mode: GRACEFUL
new_leader_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:41350
I20260709 13:10:07.862845 28603 raft_consensus.cc:606] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Received request to transfer leadership to TS 43996ee009d24ce39b66709797501e9d
I20260709 13:10:07.875461 28678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
mode: GRACEFUL
new_leader_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:10:07.875978 28678 raft_consensus.cc:606] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Received request to transfer leadership to TS 43996ee009d24ce39b66709797501e9d
I20260709 13:10:07.888674 28678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a083523f80d4904917b750d12d78d49"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
mode: GRACEFUL
new_leader_uuid: "cb418d9a7051481d976d42c15af78d0c"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:10:07.889179 28678 raft_consensus.cc:606] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Received request to transfer leadership to TS cb418d9a7051481d976d42c15af78d0c
I20260709 13:10:07.893293 28767 raft_consensus.cc:493] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.893755 28767 raft_consensus.cc:515] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.893923 28678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
mode: GRACEFUL
new_leader_uuid: "cb418d9a7051481d976d42c15af78d0c"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:10:07.894513 28678 raft_consensus.cc:606] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Received request to transfer leadership to TS cb418d9a7051481d976d42c15af78d0c
I20260709 13:10:07.895604 28767 leader_election.cc:290] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:07.896237 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:07.896634 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" is_pre_election: true
I20260709 13:10:07.896972 28603 raft_consensus.cc:2393] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43996ee009d24ce39b66709797501e9d in current term 1: Already voted for candidate f2d3a16ae3d14637b4e6aafe033a3157 in this term.
I20260709 13:10:07.898011 28493 leader_election.cc:304] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43996ee009d24ce39b66709797501e9d; no voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:07.898754 28767 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.904094 28767 raft_consensus.cc:2749] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:07.912899 28528 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6843ff6a43264ddab2199aaa87feb79b"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
mode: GRACEFUL
new_leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:34800
I20260709 13:10:07.913468 28528 raft_consensus.cc:606] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Received request to transfer leadership to TS f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:07.946501 28753 consensus_queue.cc:1048] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.959283 28757 consensus_queue.cc:1048] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:07.964468 28755 consensus_queue.cc:1048] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:07.978314 28758 consensus_queue.cc:1048] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.985932 28755 raft_consensus.cc:993] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d: : Instructing follower f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:07.986466 28755 raft_consensus.cc:1081] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Signalling peer f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:07.988005 28678 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6843ff6a43264ddab2199aaa87feb79b"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:60658
I20260709 13:10:07.988642 28678 raft_consensus.cc:493] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:07.988984 28678 raft_consensus.cc:3060] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.993955 28678 raft_consensus.cc:515] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.996153 28678 leader_election.cc:290] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 2 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:07.996460 28528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6843ff6a43264ddab2199aaa87feb79b" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:07.996789 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6843ff6a43264ddab2199aaa87feb79b" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:07.997051 28528 raft_consensus.cc:3055] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:07.997397 28528 raft_consensus.cc:740] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:07.997424 28603 raft_consensus.cc:3060] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.998301 28528 consensus_queue.cc:260] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:07.999567 28528 raft_consensus.cc:3060] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.002801 28603 raft_consensus.cc:2468] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 2.
I20260709 13:10:08.003811 28642 leader_election.cc:304] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:08.004410 28528 raft_consensus.cc:2468] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 2.
I20260709 13:10:08.004557 28725 raft_consensus.cc:2804] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.005056 28725 raft_consensus.cc:697] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Becoming Leader. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:08.005823 28725 consensus_queue.cc:237] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.011461 28427 catalog_manager.cc:5697] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 reported cstate change: term changed from 1 to 2, leader changed from 43996ee009d24ce39b66709797501e9d (127.27.63.65) to f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:08.029124 28725 raft_consensus.cc:493] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.029572 28725 raft_consensus.cc:515] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.031306 28725 leader_election.cc:290] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:08.032074 28528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43996ee009d24ce39b66709797501e9d" is_pre_election: true
I20260709 13:10:08.032372 28603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb418d9a7051481d976d42c15af78d0c" is_pre_election: true
I20260709 13:10:08.033100 28603 raft_consensus.cc:2393] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2d3a16ae3d14637b4e6aafe033a3157 in current term 1: Already voted for candidate 43996ee009d24ce39b66709797501e9d in this term.
I20260709 13:10:08.033857 28642 leader_election.cc:304] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2d3a16ae3d14637b4e6aafe033a3157; no voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c
I20260709 13:10:08.034459 28725 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:08.038910 28725 raft_consensus.cc:2749] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:08.065174 28725 consensus_queue.cc:1048] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.065814 28757 consensus_queue.cc:1048] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.073196 28737 raft_consensus.cc:993] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157: : Instructing follower cb418d9a7051481d976d42c15af78d0c to start an election
I20260709 13:10:08.073633 28737 raft_consensus.cc:1081] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Signalling peer cb418d9a7051481d976d42c15af78d0c to start an election
I20260709 13:10:08.075420 28602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a083523f80d4904917b750d12d78d49"
dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
 from {username='slave'} at 127.0.0.1:41338
I20260709 13:10:08.079332 28602 raft_consensus.cc:493] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.079774 28602 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.079929 28737 consensus_queue.cc:1048] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.083992 28722 consensus_queue.cc:1048] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.090899 28602 raft_consensus.cc:515] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.097808 28527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:08.101729 28785 raft_consensus.cc:993] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c: : Instructing follower 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:08.101577 28755 consensus_queue.cc:1048] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:08.102226 28785 raft_consensus.cc:1081] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Signalling peer 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:08.103440 28602 leader_election.cc:290] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 2 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:08.105903 28529 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:34792
I20260709 13:10:08.109397 28529 raft_consensus.cc:493] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.109819 28529 raft_consensus.cc:3060] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.107028 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:08.111784 28678 raft_consensus.cc:3055] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.112218 28678 raft_consensus.cc:740] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:08.113197 28678 consensus_queue.cc:260] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.114300 28755 consensus_queue.cc:1048] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.115175 28678 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.119855 28529 raft_consensus.cc:515] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:08.122067 28678 raft_consensus.cc:2468] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 2.
I20260709 13:10:08.123912 28567 leader_election.cc:304] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 43996ee009d24ce39b66709797501e9d
I20260709 13:10:08.124794 28753 raft_consensus.cc:2804] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.125278 28753 raft_consensus.cc:697] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Becoming Leader. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:08.126082 28753 consensus_queue.cc:237] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.128151 28529 leader_election.cc:290] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 2 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:08.129527 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:08.130167 28602 raft_consensus.cc:3055] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.130541 28602 raft_consensus.cc:740] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:08.131343 28602 consensus_queue.cc:260] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:08.131872 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddb6acaeec4670a9db1674759988df" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:08.132481 28678 raft_consensus.cc:3060] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.132715 28602 raft_consensus.cc:3060] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.133703 28737 consensus_queue.cc:1048] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.138001 28755 consensus_queue.cc:1048] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:08.140163 28602 raft_consensus.cc:2468] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 2.
I20260709 13:10:08.141371 28493 leader_election.cc:304] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:08.142277 28755 raft_consensus.cc:2804] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.142426 28678 raft_consensus.cc:2468] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 2.
I20260709 13:10:08.143059 28755 raft_consensus.cc:697] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:08.143949 28755 consensus_queue.cc:237] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } }
I20260709 13:10:08.147328 28762 raft_consensus.cc:993] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157: : Instructing follower cb418d9a7051481d976d42c15af78d0c to start an election
I20260709 13:10:08.147915 28762 raft_consensus.cc:1081] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Signalling peer cb418d9a7051481d976d42c15af78d0c to start an election
I20260709 13:10:08.150837 28603 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e"
dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
 from {username='slave'} at 127.0.0.1:41338
I20260709 13:10:08.151479 28603 raft_consensus.cc:493] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.151885 28603 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.155831 28426 catalog_manager.cc:5697] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c reported cstate change: term changed from 1 to 2, leader changed from f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67) to cb418d9a7051481d976d42c15af78d0c (127.27.63.66). New cstate: current_term: 2 leader_uuid: "cb418d9a7051481d976d42c15af78d0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.159042 28425 catalog_manager.cc:5697] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 1 to 2, leader changed from cb418d9a7051481d976d42c15af78d0c (127.27.63.66) to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 2 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:08.168022 28755 consensus_queue.cc:1048] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.173067 28603 raft_consensus.cc:515] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.173441 28725 consensus_queue.cc:1048] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.175580 28603 leader_election.cc:290] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 2 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:08.176328 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:08.177345 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:08.177961 28677 raft_consensus.cc:3055] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.178356 28677 raft_consensus.cc:740] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:08.175956 28602 raft_consensus.cc:1240] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Rejecting Update request from peer f2d3a16ae3d14637b4e6aafe033a3157 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:08.179558 28677 consensus_queue.cc:260] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.183033 28677 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.190766 28677 raft_consensus.cc:2468] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 2.
I20260709 13:10:08.191915 28567 leader_election.cc:304] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 43996ee009d24ce39b66709797501e9d
I20260709 13:10:08.192665 28753 raft_consensus.cc:2804] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.193162 28753 raft_consensus.cc:697] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Becoming Leader. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:08.193835 28753 consensus_queue.cc:237] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.200193 28426 catalog_manager.cc:5697] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c reported cstate change: term changed from 1 to 2, leader changed from f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67) to cb418d9a7051481d976d42c15af78d0c (127.27.63.66). New cstate: current_term: 2 leader_uuid: "cb418d9a7051481d976d42c15af78d0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.252585 28753 consensus_queue.cc:1048] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.261685 28785 consensus_queue.cc:1048] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.271330 28758 consensus_queue.cc:1048] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.282806 28766 consensus_queue.cc:1048] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.301995 28725 raft_consensus.cc:993] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157: : Instructing follower 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:08.302554 28797 raft_consensus.cc:1081] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Signalling peer 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:08.304237 28528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:34778
I20260709 13:10:08.304852 28528 raft_consensus.cc:493] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.305197 28528 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.309815 28528 raft_consensus.cc:515] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.311568 28528 leader_election.cc:290] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 2 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:08.312407 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:08.312669 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:08.312934 28602 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.313205 28677 raft_consensus.cc:3055] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.313521 28677 raft_consensus.cc:740] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:08.314157 28677 consensus_queue.cc:260] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.315054 28677 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.317811 28602 raft_consensus.cc:2468] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 2.
I20260709 13:10:08.318995 28493 leader_election.cc:304] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:08.319832 28766 raft_consensus.cc:2804] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.320312 28677 raft_consensus.cc:2468] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 2.
I20260709 13:10:08.320389 28766 raft_consensus.cc:697] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:08.321410 28766 consensus_queue.cc:237] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:08.327641 28427 catalog_manager.cc:5697] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 1 to 2, leader changed from f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67) to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 2 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.560981 28602 raft_consensus.cc:1275] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.562543 28797 consensus_queue.cc:1048] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.570693 28528 raft_consensus.cc:1275] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.572211 28737 consensus_queue.cc:1048] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.673897 28528 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.678804 28528 raft_consensus.cc:1275] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Refusing update from remote peer cb418d9a7051481d976d42c15af78d0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.680215 28785 consensus_queue.cc:1048] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.687803 28677 raft_consensus.cc:1275] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Refusing update from remote peer cb418d9a7051481d976d42c15af78d0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.689924 28722 consensus_queue.cc:1048] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.715821 28528 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.721094 28528 raft_consensus.cc:1275] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Refusing update from remote peer cb418d9a7051481d976d42c15af78d0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.722262 28722 consensus_queue.cc:1048] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.727193 28602 raft_consensus.cc:1275] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.730290 28766 consensus_queue.cc:1048] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.733747 28677 raft_consensus.cc:1275] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Refusing update from remote peer cb418d9a7051481d976d42c15af78d0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.736091 28604 raft_consensus.cc:1275] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.737380 28805 consensus_queue.cc:1048] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.738071 28755 consensus_queue.cc:1048] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.748006 28678 raft_consensus.cc:1275] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.750142 28767 consensus_queue.cc:1048] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.756721 28678 raft_consensus.cc:1275] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.760010 28766 consensus_queue.cc:1048] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:10.944496 27901 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:10:10.945308 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:10.964798 28529 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
mode: GRACEFUL
new_leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:34812
I20260709 13:10:10.965330 28529 raft_consensus.cc:606] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Received request to transfer leadership to TS f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:10.967769 28529 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
mode: GRACEFUL
new_leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:34812
I20260709 13:10:10.968293 28529 raft_consensus.cc:606] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Received request to transfer leadership to TS f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:10.969255 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:10.969713 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:11.274307 28806 raft_consensus.cc:993] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d: : Instructing follower f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:11.274739 28810 raft_consensus.cc:1081] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Signalling peer f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:11.276175 28677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:60658
I20260709 13:10:11.276654 28677 raft_consensus.cc:493] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:11.276955 28677 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.281944 28677 raft_consensus.cc:515] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:11.283715 28677 leader_election.cc:290] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 2 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:11.284688 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:11.284891 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:11.285233 28529 raft_consensus.cc:3055] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:11.285491 28602 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.285573 28529 raft_consensus.cc:740] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:11.286420 28529 consensus_queue.cc:260] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:11.287303 28529 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.290720 28602 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 2.
I20260709 13:10:11.291801 28529 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 2.
I20260709 13:10:11.291939 28642 leader_election.cc:304] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:11.292810 28808 raft_consensus.cc:2804] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:11.293907 28808 raft_consensus.cc:697] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Becoming Leader. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:11.294574 28808 consensus_queue.cc:237] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:11.301296 28426 catalog_manager.cc:5697] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 reported cstate change: term changed from 1 to 2, leader changed from 43996ee009d24ce39b66709797501e9d (127.27.63.65) to f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:11.464862 28810 raft_consensus.cc:993] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d: : Instructing follower f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:11.465473 28806 raft_consensus.cc:1081] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Signalling peer f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:11.466823 28677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:60658
I20260709 13:10:11.467284 28677 raft_consensus.cc:493] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:11.467576 28677 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.471843 28677 raft_consensus.cc:515] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:11.473541 28677 leader_election.cc:290] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 2 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:11.474305 28529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:11.474571 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:11.474843 28529 raft_consensus.cc:3055] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:11.475099 28602 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.475152 28529 raft_consensus.cc:740] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:11.475885 28529 consensus_queue.cc:260] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:11.476819 28529 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.479851 28602 raft_consensus.cc:2468] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 2.
I20260709 13:10:11.480917 28642 leader_election.cc:304] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:11.481473 28529 raft_consensus.cc:2468] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 2.
I20260709 13:10:11.481814 28808 raft_consensus.cc:2804] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:11.482259 28808 raft_consensus.cc:697] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Becoming Leader. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:11.482929 28808 consensus_queue.cc:237] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:11.489648 28426 catalog_manager.cc:5697] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 reported cstate change: term changed from 1 to 2, leader changed from 43996ee009d24ce39b66709797501e9d (127.27.63.65) to f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:11.714395 28529 raft_consensus.cc:1275] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.715785 28808 consensus_queue.cc:1048] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.725582 28602 raft_consensus.cc:1275] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.727254 28808 consensus_queue.cc:1048] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.914723 28604 raft_consensus.cc:1275] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.916620 28808 consensus_queue.cc:1048] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:11.929036 28529 raft_consensus.cc:1275] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.931063 28808 consensus_queue.cc:1048] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.970768 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:11.988849 28677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
mode: GRACEFUL
new_leader_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:10:11.989471 28677 raft_consensus.cc:606] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Received request to transfer leadership to TS 43996ee009d24ce39b66709797501e9d
I20260709 13:10:11.998359 28677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
mode: GRACEFUL
new_leader_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:10:11.998917 28677 raft_consensus.cc:606] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Received request to transfer leadership to TS 43996ee009d24ce39b66709797501e9d
I20260709 13:10:12.003921 28604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a083523f80d4904917b750d12d78d49"
dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
mode: GRACEFUL
new_leader_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:41350
I20260709 13:10:12.004463 28604 raft_consensus.cc:606] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Received request to transfer leadership to TS 43996ee009d24ce39b66709797501e9d
I20260709 13:10:12.009464 28604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e"
dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
mode: GRACEFUL
new_leader_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:41350
I20260709 13:10:12.010105 28604 raft_consensus.cc:606] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Received request to transfer leadership to TS 43996ee009d24ce39b66709797501e9d
I20260709 13:10:12.015439 28604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20c3599ec7274ec59d965dac9cffcacb"
dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
mode: GRACEFUL
new_leader_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:41350
I20260709 13:10:12.015908 28604 raft_consensus.cc:606] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Received request to transfer leadership to TS 43996ee009d24ce39b66709797501e9d
I20260709 13:10:12.157507 28809 raft_consensus.cc:993] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c: : Instructing follower 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.157965 28809 raft_consensus.cc:1081] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Signalling peer 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.159689 28529 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:34792
I20260709 13:10:12.160238 28529 raft_consensus.cc:493] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:12.160579 28529 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.166191 28529 raft_consensus.cc:515] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.167932 28529 leader_election.cc:290] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 3 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:12.168968 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:12.168936 28604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:12.169718 28677 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.169804 28604 raft_consensus.cc:3055] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:12.170187 28604 raft_consensus.cc:740] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:12.170915 28604 consensus_queue.cc:260] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.171836 28604 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.174620 28677 raft_consensus.cc:2468] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 3.
I20260709 13:10:12.175840 28492 leader_election.cc:304] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:12.176776 28806 raft_consensus.cc:2804] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:12.177212 28806 raft_consensus.cc:697] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:12.177470 28604 raft_consensus.cc:2468] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 3.
I20260709 13:10:12.178158 28806 consensus_queue.cc:237] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.185163 28425 catalog_manager.cc:5697] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 2 to 3, leader changed from cb418d9a7051481d976d42c15af78d0c (127.27.63.66) to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 3 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:12.238924 28809 raft_consensus.cc:993] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c: : Instructing follower 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.239444 28820 raft_consensus.cc:1081] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Signalling peer 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.240861 28529 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a083523f80d4904917b750d12d78d49"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:34792
I20260709 13:10:12.241374 28529 raft_consensus.cc:493] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:12.241701 28529 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.246815 28529 raft_consensus.cc:515] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.248384 28529 leader_election.cc:290] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 3 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:12.249356 28604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:12.249414 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:12.249878 28604 raft_consensus.cc:3055] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:12.250003 28677 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.250187 28604 raft_consensus.cc:740] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:12.250916 28604 consensus_queue.cc:260] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.251817 28604 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.254235 28677 raft_consensus.cc:2468] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 3.
I20260709 13:10:12.255205 28492 leader_election.cc:304] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:12.256121 28806 raft_consensus.cc:2804] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:12.256621 28806 raft_consensus.cc:697] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:12.256793 28604 raft_consensus.cc:2468] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 3.
I20260709 13:10:12.257720 28806 consensus_queue.cc:237] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.265095 28425 catalog_manager.cc:5697] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 2 to 3, leader changed from cb418d9a7051481d976d42c15af78d0c (127.27.63.66) to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 3 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:12.318313 28809 raft_consensus.cc:993] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c: : Instructing follower 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.318800 28809 raft_consensus.cc:1081] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Signalling peer 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.320382 28529 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20c3599ec7274ec59d965dac9cffcacb"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:34792
I20260709 13:10:12.320930 28529 raft_consensus.cc:493] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:12.321275 28529 raft_consensus.cc:3060] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.323187 28828 raft_consensus.cc:993] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157: : Instructing follower 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.323632 28828 raft_consensus.cc:1081] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Signalling peer 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.325346 28528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:34778
I20260709 13:10:12.325879 28528 raft_consensus.cc:493] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:12.326218 28528 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.327761 28529 raft_consensus.cc:515] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.329967 28529 leader_election.cc:290] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 2 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:12.331327 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20c3599ec7274ec59d965dac9cffcacb" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:12.331327 28604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20c3599ec7274ec59d965dac9cffcacb" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:12.332085 28677 raft_consensus.cc:3060] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.332187 28604 raft_consensus.cc:3055] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:12.332530 28604 raft_consensus.cc:740] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:12.332901 28528 raft_consensus.cc:515] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.333372 28604 consensus_queue.cc:260] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.334824 28528 leader_election.cc:290] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 3 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:12.335762 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:12.335934 28604 raft_consensus.cc:3060] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:12.336395 28602 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.336596 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:12.337160 28678 raft_consensus.cc:3055] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:12.337455 28678 raft_consensus.cc:740] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:12.338300 28678 consensus_queue.cc:260] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.339682 28678 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.339934 28677 raft_consensus.cc:2468] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 2.
I20260709 13:10:12.342132 28604 raft_consensus.cc:2468] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 2.
I20260709 13:10:12.343312 28493 leader_election.cc:304] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:12.344100 28806 raft_consensus.cc:2804] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:12.344444 28602 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 3.
I20260709 13:10:12.344605 28806 raft_consensus.cc:697] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:12.345546 28806 consensus_queue.cc:237] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.347303 28678 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 3.
I20260709 13:10:12.348357 28492 leader_election.cc:304] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:12.349442 28810 raft_consensus.cc:2804] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:12.350114 28810 raft_consensus.cc:697] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:12.351281 28810 consensus_queue.cc:237] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.355506 28425 catalog_manager.cc:5697] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 1 to 2, leader changed from cb418d9a7051481d976d42c15af78d0c (127.27.63.66) to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 2 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:12.367048 28425 catalog_manager.cc:5697] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 2 to 3, leader changed from f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67) to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 3 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:12.513540 28828 raft_consensus.cc:993] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157: : Instructing follower 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.514089 28808 raft_consensus.cc:1081] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Signalling peer 43996ee009d24ce39b66709797501e9d to start an election
I20260709 13:10:12.515445 28528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
 from {username='slave'} at 127.0.0.1:34778
I20260709 13:10:12.515914 28528 raft_consensus.cc:493] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:12.516252 28528 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.521420 28528 raft_consensus.cc:515] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.522879 28528 leader_election.cc:290] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 3 election: Requested vote from peers cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:12.523674 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:12.523854 28678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31a71981d19c44d0be6a1d04a520c9c1" candidate_uuid: "43996ee009d24ce39b66709797501e9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:12.524160 28602 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.524478 28678 raft_consensus.cc:3055] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:12.524812 28678 raft_consensus.cc:740] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:12.525470 28678 consensus_queue.cc:260] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.526388 28678 raft_consensus.cc:3060] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.529055 28602 raft_consensus.cc:2468] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 3.
I20260709 13:10:12.530006 28493 leader_election.cc:304] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43996ee009d24ce39b66709797501e9d, cb418d9a7051481d976d42c15af78d0c; no voters: 
I20260709 13:10:12.530820 28810 raft_consensus.cc:2804] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:12.531270 28810 raft_consensus.cc:697] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Becoming Leader. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:12.532102 28678 raft_consensus.cc:2468] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43996ee009d24ce39b66709797501e9d in term 3.
I20260709 13:10:12.532152 28810 consensus_queue.cc:237] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:12.538344 28425 catalog_manager.cc:5697] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d reported cstate change: term changed from 2 to 3, leader changed from f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67) to 43996ee009d24ce39b66709797501e9d (127.27.63.65). New cstate: current_term: 3 leader_uuid: "43996ee009d24ce39b66709797501e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:12.623955 28602 raft_consensus.cc:1275] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:12.625226 28810 consensus_queue.cc:1048] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:12.638302 28678 raft_consensus.cc:1275] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:12.639802 28836 consensus_queue.cc:1048] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:12.696431 28602 raft_consensus.cc:1275] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:12.697589 28836 consensus_queue.cc:1048] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:12.707278 28678 raft_consensus.cc:1275] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:12.708900 28843 consensus_queue.cc:1048] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:12.730827 28602 raft_consensus.cc:1275] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:12.731978 28843 consensus_queue.cc:1048] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:12.742197 28678 raft_consensus.cc:1275] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:12.743528 28843 consensus_queue.cc:1048] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:12.746610 28677 raft_consensus.cc:1275] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:12.747941 28806 consensus_queue.cc:1048] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:12.756589 28602 raft_consensus.cc:1275] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:12.758188 28843 consensus_queue.cc:1048] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:13.125643 28602 raft_consensus.cc:1275] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:13.126842 28843 consensus_queue.cc:1048] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:13.137771 28678 raft_consensus.cc:1275] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Refusing update from remote peer 43996ee009d24ce39b66709797501e9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:13.139286 28843 consensus_queue.cc:1048] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:15.023357 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:15.031548 28528 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
mode: GRACEFUL
new_leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:34812
I20260709 13:10:15.032138 28528 raft_consensus.cc:606] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Received request to transfer leadership to TS f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:15.034595 28528 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
mode: GRACEFUL
new_leader_uuid: "cb418d9a7051481d976d42c15af78d0c"
 from {username='slave'} at 127.0.0.1:34812
I20260709 13:10:15.035181 28528 raft_consensus.cc:606] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Received request to transfer leadership to TS cb418d9a7051481d976d42c15af78d0c
I20260709 13:10:15.037446 28528 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a083523f80d4904917b750d12d78d49"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
mode: GRACEFUL
new_leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:34812
I20260709 13:10:15.037916 28528 raft_consensus.cc:606] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Received request to transfer leadership to TS f2d3a16ae3d14637b4e6aafe033a3157
I20260709 13:10:15.040308 28528 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33"
dest_uuid: "43996ee009d24ce39b66709797501e9d"
mode: GRACEFUL
new_leader_uuid: "cb418d9a7051481d976d42c15af78d0c"
 from {username='slave'} at 127.0.0.1:34812
I20260709 13:10:15.040922 28528 raft_consensus.cc:606] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Received request to transfer leadership to TS cb418d9a7051481d976d42c15af78d0c
I20260709 13:10:15.042145 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:10:15.042678 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:15.156306 28851 raft_consensus.cc:993] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d: : Instructing follower cb418d9a7051481d976d42c15af78d0c to start an election
I20260709 13:10:15.156773 28806 raft_consensus.cc:1081] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Signalling peer cb418d9a7051481d976d42c15af78d0c to start an election
I20260709 13:10:15.158227 28602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33"
dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:10:15.158600 28602 raft_consensus.cc:493] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:15.158903 28602 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.163937 28602 raft_consensus.cc:515] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.165455 28602 leader_election.cc:290] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 4 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:15.166451 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:15.166358 28528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dce59ce0034d3d8351861f2ff79b33" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:15.166982 28677 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.167083 28528 raft_consensus.cc:3055] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:15.167420 28528 raft_consensus.cc:740] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:15.168141 28528 consensus_queue.cc:260] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.169018 28528 raft_consensus.cc:3060] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.171196 28677 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 4.
I20260709 13:10:15.172333 28567 leader_election.cc:304] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:15.173111 28852 raft_consensus.cc:2804] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:15.173524 28528 raft_consensus.cc:2468] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 4.
I20260709 13:10:15.174278 28852 raft_consensus.cc:697] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 4 LEADER]: Becoming Leader. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:15.175149 28852 consensus_queue.cc:237] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.177536 28851 raft_consensus.cc:993] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d: : Instructing follower f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:15.178045 28806 raft_consensus.cc:1081] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Signalling peer f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:15.179469 28677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a083523f80d4904917b750d12d78d49"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:60658
I20260709 13:10:15.179878 28677 raft_consensus.cc:493] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:15.180172 28677 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.182232 28425 catalog_manager.cc:5697] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c reported cstate change: term changed from 3 to 4, leader changed from 43996ee009d24ce39b66709797501e9d (127.27.63.65) to cb418d9a7051481d976d42c15af78d0c (127.27.63.66). New cstate: current_term: 4 leader_uuid: "cb418d9a7051481d976d42c15af78d0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:15.184034 28677 raft_consensus.cc:515] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.185552 28677 leader_election.cc:290] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 4 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:15.186604 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:15.186851 28528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a083523f80d4904917b750d12d78d49" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:15.187135 28602 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.187397 28528 raft_consensus.cc:3055] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:15.187880 28528 raft_consensus.cc:740] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:15.188659 28528 consensus_queue.cc:260] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.189451 28528 raft_consensus.cc:3060] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.193420 28602 raft_consensus.cc:2468] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 4.
I20260709 13:10:15.194777 28642 leader_election.cc:304] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:15.195487 28528 raft_consensus.cc:2468] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 4.
I20260709 13:10:15.195606 28808 raft_consensus.cc:2804] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:15.196849 28808 raft_consensus.cc:697] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 LEADER]: Becoming Leader. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:15.197659 28808 consensus_queue.cc:237] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.203685 28425 catalog_manager.cc:5697] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 reported cstate change: term changed from 3 to 4, leader changed from 43996ee009d24ce39b66709797501e9d (127.27.63.65) to f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67). New cstate: current_term: 4 leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:15.231256 28851 raft_consensus.cc:993] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d: : Instructing follower cb418d9a7051481d976d42c15af78d0c to start an election
I20260709 13:10:15.231727 28851 raft_consensus.cc:1081] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Signalling peer cb418d9a7051481d976d42c15af78d0c to start an election
I20260709 13:10:15.233161 28602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e"
dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
 from {username='slave'} at 127.0.0.1:41342
I20260709 13:10:15.233610 28602 raft_consensus.cc:493] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:15.233893 28602 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.238538 28602 raft_consensus.cc:515] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.240262 28602 leader_election.cc:290] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 4 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67:38283)
I20260709 13:10:15.241201 28677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
I20260709 13:10:15.240871 28528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1bb38c07c45efa9d1694302f4853e" candidate_uuid: "cb418d9a7051481d976d42c15af78d0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:15.241632 28677 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.241972 28528 raft_consensus.cc:3055] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:15.242250 28528 raft_consensus.cc:740] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:15.242784 28528 consensus_queue.cc:260] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.243611 28528 raft_consensus.cc:3060] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.247061 28677 raft_consensus.cc:2468] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 4.
I20260709 13:10:15.247916 28567 leader_election.cc:304] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:15.248572 28852 raft_consensus.cc:2804] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:15.249053 28852 raft_consensus.cc:697] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 4 LEADER]: Becoming Leader. State: Replica: cb418d9a7051481d976d42c15af78d0c, State: Running, Role: LEADER
I20260709 13:10:15.249233 28528 raft_consensus.cc:2468] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb418d9a7051481d976d42c15af78d0c in term 4.
I20260709 13:10:15.249887 28852 consensus_queue.cc:237] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.255932 28425 catalog_manager.cc:5697] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c reported cstate change: term changed from 3 to 4, leader changed from 43996ee009d24ce39b66709797501e9d (127.27.63.65) to cb418d9a7051481d976d42c15af78d0c (127.27.63.66). New cstate: current_term: 4 leader_uuid: "cb418d9a7051481d976d42c15af78d0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:15.433015 28806 raft_consensus.cc:993] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d: : Instructing follower f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:15.433467 28851 raft_consensus.cc:1081] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Signalling peer f2d3a16ae3d14637b4e6aafe033a3157 to start an election
I20260709 13:10:15.434787 28677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff"
dest_uuid: "f2d3a16ae3d14637b4e6aafe033a3157"
 from {username='slave'} at 127.0.0.1:60658
I20260709 13:10:15.435278 28677 raft_consensus.cc:493] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:15.435520 28677 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:15.439698 28677 raft_consensus.cc:515] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.441152 28677 leader_election.cc:290] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 3 election: Requested vote from peers 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821), cb418d9a7051481d976d42c15af78d0c (127.27.63.66:36721)
I20260709 13:10:15.442067 28528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "43996ee009d24ce39b66709797501e9d"
I20260709 13:10:15.442215 28602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171aa8eb9fb74f928c1abcf41b8a72ff" candidate_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb418d9a7051481d976d42c15af78d0c"
I20260709 13:10:15.442566 28528 raft_consensus.cc:3055] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:15.442713 28602 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:15.442859 28528 raft_consensus.cc:740] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 43996ee009d24ce39b66709797501e9d, State: Running, Role: LEADER
I20260709 13:10:15.443666 28528 consensus_queue.cc:260] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.444777 28528 raft_consensus.cc:3060] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:15.448482 28602 raft_consensus.cc:2468] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 3.
I20260709 13:10:15.449361 28528 raft_consensus.cc:2468] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d3a16ae3d14637b4e6aafe033a3157 in term 3.
I20260709 13:10:15.449510 28642 leader_election.cc:304] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb418d9a7051481d976d42c15af78d0c, f2d3a16ae3d14637b4e6aafe033a3157; no voters: 
I20260709 13:10:15.450438 28808 raft_consensus.cc:2804] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:15.450882 28808 raft_consensus.cc:697] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 LEADER]: Becoming Leader. State: Replica: f2d3a16ae3d14637b4e6aafe033a3157, State: Running, Role: LEADER
I20260709 13:10:15.451637 28808 consensus_queue.cc:237] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } }
I20260709 13:10:15.458536 28425 catalog_manager.cc:5697] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 reported cstate change: term changed from 2 to 3, leader changed from 43996ee009d24ce39b66709797501e9d (127.27.63.65) to f2d3a16ae3d14637b4e6aafe033a3157 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:15.556463 28528 raft_consensus.cc:1275] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Refusing update from remote peer cb418d9a7051481d976d42c15af78d0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:15.557643 28852 consensus_queue.cc:1048] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:15.567988 28677 raft_consensus.cc:1275] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Refusing update from remote peer cb418d9a7051481d976d42c15af78d0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:15.570998 28852 consensus_queue.cc:1048] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:15.587837 28602 raft_consensus.cc:1275] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:15.588932 28808 consensus_queue.cc:1048] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:15.597532 28528 raft_consensus.cc:1275] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:15.599296 28808 consensus_queue.cc:1048] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:15.649926 28528 raft_consensus.cc:1275] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Refusing update from remote peer cb418d9a7051481d976d42c15af78d0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:15.651032 28852 consensus_queue.cc:1048] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:15.658880 28677 raft_consensus.cc:1275] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Refusing update from remote peer cb418d9a7051481d976d42c15af78d0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:15.660471 28852 consensus_queue.cc:1048] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d3a16ae3d14637b4e6aafe033a3157" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 38283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:15.958468 28528 raft_consensus.cc:1275] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:15.959590 28808 consensus_queue.cc:1048] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43996ee009d24ce39b66709797501e9d" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 39821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:15.969043 28602 raft_consensus.cc:1275] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Refusing update from remote peer f2d3a16ae3d14637b4e6aafe033a3157: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:15.970634 28875 consensus_queue.cc:1048] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb418d9a7051481d976d42c15af78d0c" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:16.043529 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:16.049777 27901 tablet_server.cc:179] TabletServer@127.27.63.65:0 shutting down...
W20260709 13:10:16.061085 28569 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 36 similar messages]
W20260709 13:10:16.067219 28569 consensus_peers.cc:597] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c -> Peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Couldn't send request to peer 43996ee009d24ce39b66709797501e9d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:16.086951 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:16.087652 27901 tablet_replica.cc:333] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.088317 27901 raft_consensus.cc:2243] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.088783 27901 raft_consensus.cc:2272] T 171aa8eb9fb74f928c1abcf41b8a72ff P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.090988 27901 tablet_replica.cc:333] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.091490 27901 raft_consensus.cc:2243] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.091872 27901 raft_consensus.cc:2272] T 7a083523f80d4904917b750d12d78d49 P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.093874 27901 tablet_replica.cc:333] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.094369 27901 raft_consensus.cc:2243] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.095027 27901 raft_consensus.cc:2272] T 20c3599ec7274ec59d965dac9cffcacb P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.096655 27901 tablet_replica.cc:333] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.097193 27901 raft_consensus.cc:2243] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.097586 27901 raft_consensus.cc:2272] T 59d1bb38c07c45efa9d1694302f4853e P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.099490 27901 tablet_replica.cc:333] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.100018 27901 raft_consensus.cc:2243] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.100485 27901 raft_consensus.cc:2272] T 2ea28f8b3c534695b1888ed36e8fe6ac P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.102030 27901 tablet_replica.cc:333] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.102505 27901 raft_consensus.cc:2243] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.103171 27901 raft_consensus.cc:2272] T 31a71981d19c44d0be6a1d04a520c9c1 P 43996ee009d24ce39b66709797501e9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.104779 27901 tablet_replica.cc:333] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.105252 27901 raft_consensus.cc:2243] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.105962 27901 raft_consensus.cc:2272] T 20ddb6acaeec4670a9db1674759988df P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.107599 27901 tablet_replica.cc:333] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.108122 27901 raft_consensus.cc:2243] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.108492 27901 raft_consensus.cc:2272] T c6dce59ce0034d3d8351861f2ff79b33 P 43996ee009d24ce39b66709797501e9d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.110298 27901 tablet_replica.cc:333] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.110760 27901 raft_consensus.cc:2243] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.111186 27901 raft_consensus.cc:2272] T 6843ff6a43264ddab2199aaa87feb79b P 43996ee009d24ce39b66709797501e9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.112859 27901 tablet_replica.cc:333] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d: stopping tablet replica
I20260709 13:10:16.113277 27901 raft_consensus.cc:2243] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.113972 27901 raft_consensus.cc:2272] T 308591aaed574bc5a1e9391c834f01dc P 43996ee009d24ce39b66709797501e9d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:16.123350 28643 consensus_peers.cc:597] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 -> Peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Couldn't send request to peer 43996ee009d24ce39b66709797501e9d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:16.136245 27901 tablet_server.cc:196] TabletServer@127.27.63.65:0 shutdown complete.
W20260709 13:10:16.139577 28643 consensus_peers.cc:597] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 -> Peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Couldn't send request to peer 43996ee009d24ce39b66709797501e9d. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:39821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:16.139779 28569 consensus_peers.cc:597] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c -> Peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Couldn't send request to peer 43996ee009d24ce39b66709797501e9d. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:39821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:16.157236 27901 tablet_server.cc:179] TabletServer@127.27.63.66:0 shutting down...
W20260709 13:10:16.169195 28569 consensus_peers.cc:597] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c -> Peer 43996ee009d24ce39b66709797501e9d (127.27.63.65:39821): Couldn't send request to peer 43996ee009d24ce39b66709797501e9d. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:39821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:16.190706 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:16.191392 27901 tablet_replica.cc:333] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.191959 27901 raft_consensus.cc:2243] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.192767 27901 raft_consensus.cc:2272] T 59d1bb38c07c45efa9d1694302f4853e P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.194877 27901 tablet_replica.cc:333] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.195365 27901 raft_consensus.cc:2243] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.195786 27901 raft_consensus.cc:2272] T 6843ff6a43264ddab2199aaa87feb79b P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.197499 27901 tablet_replica.cc:333] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.198006 27901 raft_consensus.cc:2243] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.198379 27901 raft_consensus.cc:2272] T 20c3599ec7274ec59d965dac9cffcacb P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.199900 27901 tablet_replica.cc:333] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.200402 27901 raft_consensus.cc:2243] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.200778 27901 raft_consensus.cc:2272] T 7a083523f80d4904917b750d12d78d49 P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.203157 27901 tablet_replica.cc:333] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.203615 27901 raft_consensus.cc:2243] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.203971 27901 raft_consensus.cc:2272] T 308591aaed574bc5a1e9391c834f01dc P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.205559 27901 tablet_replica.cc:333] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.205971 27901 raft_consensus.cc:2243] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.206301 27901 raft_consensus.cc:2272] T 20ddb6acaeec4670a9db1674759988df P cb418d9a7051481d976d42c15af78d0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.207773 27901 tablet_replica.cc:333] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.208250 27901 raft_consensus.cc:2243] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.208871 27901 raft_consensus.cc:2272] T c6dce59ce0034d3d8351861f2ff79b33 P cb418d9a7051481d976d42c15af78d0c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.210602 27901 tablet_replica.cc:333] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.211068 27901 raft_consensus.cc:2243] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.211730 27901 raft_consensus.cc:2272] T 2ea28f8b3c534695b1888ed36e8fe6ac P cb418d9a7051481d976d42c15af78d0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.213296 27901 tablet_replica.cc:333] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.213670 27901 raft_consensus.cc:2243] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.214007 27901 raft_consensus.cc:2272] T 31a71981d19c44d0be6a1d04a520c9c1 P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.215469 27901 tablet_replica.cc:333] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c: stopping tablet replica
I20260709 13:10:16.215850 27901 raft_consensus.cc:2243] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.216245 27901 raft_consensus.cc:2272] T 171aa8eb9fb74f928c1abcf41b8a72ff P cb418d9a7051481d976d42c15af78d0c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.243733 27901 tablet_server.cc:196] TabletServer@127.27.63.66:0 shutdown complete.
I20260709 13:10:16.261281 27901 tablet_server.cc:179] TabletServer@127.27.63.67:0 shutting down...
I20260709 13:10:16.281657 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:16.282435 27901 tablet_replica.cc:333] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.282985 27901 raft_consensus.cc:2243] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.283684 27901 raft_consensus.cc:2272] T 7a083523f80d4904917b750d12d78d49 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.285691 27901 tablet_replica.cc:333] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.286162 27901 raft_consensus.cc:2243] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.286522 27901 raft_consensus.cc:2272] T 59d1bb38c07c45efa9d1694302f4853e P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.288506 27901 tablet_replica.cc:333] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.289003 27901 raft_consensus.cc:2243] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.289637 27901 raft_consensus.cc:2272] T 6843ff6a43264ddab2199aaa87feb79b P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.291080 27901 tablet_replica.cc:333] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.291517 27901 raft_consensus.cc:2243] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.291854 27901 raft_consensus.cc:2272] T 308591aaed574bc5a1e9391c834f01dc P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.293423 27901 tablet_replica.cc:333] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.293828 27901 raft_consensus.cc:2243] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.294206 27901 raft_consensus.cc:2272] T c6dce59ce0034d3d8351861f2ff79b33 P f2d3a16ae3d14637b4e6aafe033a3157 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.296223 27901 tablet_replica.cc:333] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.296671 27901 raft_consensus.cc:2243] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.297017 27901 raft_consensus.cc:2272] T 20ddb6acaeec4670a9db1674759988df P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.298848 27901 tablet_replica.cc:333] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.299314 27901 raft_consensus.cc:2243] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.299692 27901 raft_consensus.cc:2272] T 31a71981d19c44d0be6a1d04a520c9c1 P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.301391 27901 tablet_replica.cc:333] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.301841 27901 raft_consensus.cc:2243] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.302189 27901 raft_consensus.cc:2272] T 2ea28f8b3c534695b1888ed36e8fe6ac P f2d3a16ae3d14637b4e6aafe033a3157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.303859 27901 tablet_replica.cc:333] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.304355 27901 raft_consensus.cc:2243] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:16.304723 27901 raft_consensus.cc:2272] T 20c3599ec7274ec59d965dac9cffcacb P f2d3a16ae3d14637b4e6aafe033a3157 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.306192 27901 tablet_replica.cc:333] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157: stopping tablet replica
I20260709 13:10:16.306594 27901 raft_consensus.cc:2243] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.307273 27901 raft_consensus.cc:2272] T 171aa8eb9fb74f928c1abcf41b8a72ff P f2d3a16ae3d14637b4e6aafe033a3157 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.327569 27901 tablet_server.cc:196] TabletServer@127.27.63.67:0 shutdown complete.
I20260709 13:10:16.347503 27901 master.cc:562] Master@127.27.63.126:35637 shutting down...
I20260709 13:10:16.367029 27901 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:16.367667 27901 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:16.368175 27901 tablet_replica.cc:333] T 00000000000000000000000000000000 P 67a71f07f30e4694b2e49d9516f84df7: stopping tablet replica
I20260709 13:10:16.395947 27901 master.cc:584] Master@127.27.63.126:35637 shutdown complete.
43996ee009d24ce39b66709797501e9d  5
cb418d9a7051481d976d42c15af78d0c  4
f2d3a16ae3d14637b4e6aafe033a3157  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10129 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:10:16.437275 27901 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.63.126:43467
I20260709 13:10:16.438367 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:16.444443 28884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.444653 28883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.444715 28886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:16.445785 27901 server_base.cc:1061] running on GCE node
I20260709 13:10:16.446610 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:16.446830 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:16.446986 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602616446962 us; error 0 us; skew 500 ppm
I20260709 13:10:16.447487 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:16.450105 27901 webserver.cc:533] Webserver started at http://127.27.63.126:37677/ using document root <none> and password file <none>
I20260709 13:10:16.450575 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:16.450766 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:16.451048 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:16.452196 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/master-0-root/instance:
uuid: "c44d1d5117734537b7202f74e0c1e920"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-sjgx"
I20260709 13:10:16.456540 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:10:16.459782 28891 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.460569 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:10:16.460876 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/master-0-root
uuid: "c44d1d5117734537b7202f74e0c1e920"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-sjgx"
I20260709 13:10:16.461155 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:16.482641 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:16.483878 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:16.520864 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.126:43467
I20260709 13:10:16.520959 28942 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.126:43467 every 8 connection(s)
I20260709 13:10:16.524700 28943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:16.534085 28943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Bootstrap starting.
I20260709 13:10:16.538511 28943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:16.542850 28943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: No bootstrap required, opened a new log
I20260709 13:10:16.544915 28943 raft_consensus.cc:359] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44d1d5117734537b7202f74e0c1e920" member_type: VOTER }
I20260709 13:10:16.545284 28943 raft_consensus.cc:385] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:16.545455 28943 raft_consensus.cc:740] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c44d1d5117734537b7202f74e0c1e920, State: Initialized, Role: FOLLOWER
I20260709 13:10:16.545990 28943 consensus_queue.cc:260] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44d1d5117734537b7202f74e0c1e920" member_type: VOTER }
I20260709 13:10:16.546399 28943 raft_consensus.cc:399] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:16.546586 28943 raft_consensus.cc:493] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:16.546802 28943 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:16.551028 28943 raft_consensus.cc:515] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44d1d5117734537b7202f74e0c1e920" member_type: VOTER }
I20260709 13:10:16.551517 28943 leader_election.cc:304] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c44d1d5117734537b7202f74e0c1e920; no voters: 
I20260709 13:10:16.552608 28943 leader_election.cc:290] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:16.552960 28946 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:16.554303 28946 raft_consensus.cc:697] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 1 LEADER]: Becoming Leader. State: Replica: c44d1d5117734537b7202f74e0c1e920, State: Running, Role: LEADER
I20260709 13:10:16.554991 28946 consensus_queue.cc:237] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44d1d5117734537b7202f74e0c1e920" member_type: VOTER }
I20260709 13:10:16.555531 28943 sys_catalog.cc:565] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:16.560086 28947 sys_catalog.cc:455] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c44d1d5117734537b7202f74e0c1e920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44d1d5117734537b7202f74e0c1e920" member_type: VOTER } }
I20260709 13:10:16.560472 28948 sys_catalog.cc:455] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c44d1d5117734537b7202f74e0c1e920. Latest consensus state: current_term: 1 leader_uuid: "c44d1d5117734537b7202f74e0c1e920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44d1d5117734537b7202f74e0c1e920" member_type: VOTER } }
I20260709 13:10:16.560915 28947 sys_catalog.cc:458] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:16.561194 28948 sys_catalog.cc:458] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:16.564296 28954 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:16.569046 28954 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:16.571931 27901 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:16.577703 28954 catalog_manager.cc:1383] Generated new cluster ID: 2a6d06a3b3ea405e84dd889e97329460
I20260709 13:10:16.577958 28954 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:16.608420 28954 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:16.609833 28954 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:16.628763 28954 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Generated new TSK 0
I20260709 13:10:16.629487 28954 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:16.639974 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:16.646231 28964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.647478 28965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:16.649751 27901 server_base.cc:1061] running on GCE node
W20260709 13:10:16.650180 28967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:16.651084 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:16.651304 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:16.651472 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602616651457 us; error 0 us; skew 500 ppm
I20260709 13:10:16.652024 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:16.654421 27901 webserver.cc:533] Webserver started at http://127.27.63.65:37577/ using document root <none> and password file <none>
I20260709 13:10:16.654923 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:16.655083 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:16.655408 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:16.656549 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/instance:
uuid: "06ae4197fd784525b695a9b16c8094ea"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-sjgx"
I20260709 13:10:16.661038 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:10:16.664526 28972 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.665295 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.002s
I20260709 13:10:16.665597 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "06ae4197fd784525b695a9b16c8094ea"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-sjgx"
I20260709 13:10:16.666014 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:16.684170 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:16.685379 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:16.686841 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:16.689152 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:16.689388 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.689626 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:16.689807 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.730310 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:43941
I20260709 13:10:16.730443 29034 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:43941 every 8 connection(s)
I20260709 13:10:16.735239 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:16.742713 29039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.744505 29040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.749569 29042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:16.750798 27901 server_base.cc:1061] running on GCE node
I20260709 13:10:16.752110 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:16.752410 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:16.752624 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602616752607 us; error 0 us; skew 500 ppm
I20260709 13:10:16.753404 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:16.754892 29035 heartbeater.cc:344] Connected to a master server at 127.27.63.126:43467
I20260709 13:10:16.755280 29035 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:16.756213 29035 heartbeater.cc:507] Master 127.27.63.126:43467 requested a full tablet report, sending...
I20260709 13:10:16.756494 27901 webserver.cc:533] Webserver started at http://127.27.63.66:43691/ using document root <none> and password file <none>
I20260709 13:10:16.757122 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:16.757300 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:16.757637 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:16.758600 28908 ts_manager.cc:194] Registered new tserver with Master: 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:10:16.758834 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/instance:
uuid: "befda4a6c8344c26b0f139e4e1686b3b"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-sjgx"
I20260709 13:10:16.760684 28908 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52660
I20260709 13:10:16.764010 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:10:16.767499 29047 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.768563 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:10:16.768841 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root
uuid: "befda4a6c8344c26b0f139e4e1686b3b"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-sjgx"
I20260709 13:10:16.769084 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:16.787220 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:16.788460 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:16.789832 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:16.792006 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:16.792280 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.792546 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:16.792706 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.832070 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.66:35939
I20260709 13:10:16.832286 29109 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.66:35939 every 8 connection(s)
I20260709 13:10:16.836709 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:16.844316 29113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.844866 29114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:16.848443 29117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:16.848652 27901 server_base.cc:1061] running on GCE node
I20260709 13:10:16.850049 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:10:16.850383 29110 heartbeater.cc:344] Connected to a master server at 127.27.63.126:43467
W20260709 13:10:16.850384 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:16.850886 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602616850867 us; error 0 us; skew 500 ppm
I20260709 13:10:16.850948 29110 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:16.851567 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:16.851816 29110 heartbeater.cc:507] Master 127.27.63.126:43467 requested a full tablet report, sending...
I20260709 13:10:16.853935 28908 ts_manager.cc:194] Registered new tserver with Master: befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:16.854400 27901 webserver.cc:533] Webserver started at http://127.27.63.67:43975/ using document root <none> and password file <none>
I20260709 13:10:16.855000 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:16.855232 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:16.855404 28908 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52674
I20260709 13:10:16.855530 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:16.857156 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/instance:
uuid: "243d4ae4a4ae48b2979e32874472d256"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-sjgx"
I20260709 13:10:16.861835 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:10:16.865429 29121 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.866321 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:10:16.866639 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root
uuid: "243d4ae4a4ae48b2979e32874472d256"
format_stamp: "Formatted at 2026-07-09 13:10:16 on dist-test-slave-sjgx"
I20260709 13:10:16.867014 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:16.885588 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:16.886981 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:16.888638 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:16.891004 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:16.891258 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.891530 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:16.891728 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:16.932902 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.67:40957
I20260709 13:10:16.932991 29183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.67:40957 every 8 connection(s)
I20260709 13:10:16.946640 29184 heartbeater.cc:344] Connected to a master server at 127.27.63.126:43467
I20260709 13:10:16.946962 29184 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:16.947643 29184 heartbeater.cc:507] Master 127.27.63.126:43467 requested a full tablet report, sending...
I20260709 13:10:16.949472 28908 ts_manager.cc:194] Registered new tserver with Master: 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:16.949890 27901 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013222456s
I20260709 13:10:16.950845 28908 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52676
I20260709 13:10:16.975293 28908 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52684:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:16.981367 28908 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:17.175779 29075 tablet_service.cc:1511] Processing CreateTablet for tablet 7216d7dc49494630845e9eeb30dc6902 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:17.177320 29075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7216d7dc49494630845e9eeb30dc6902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.179204 29000 tablet_service.cc:1511] Processing CreateTablet for tablet 7216d7dc49494630845e9eeb30dc6902 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:17.181222 28999 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:17.181284 28997 tablet_service.cc:1511] Processing CreateTablet for tablet e583be49bed64dd2996afcdcffa95120 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:17.182780 28999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.184072 29072 tablet_service.cc:1511] Processing CreateTablet for tablet e583be49bed64dd2996afcdcffa95120 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:17.185361 29072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e583be49bed64dd2996afcdcffa95120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.186995 28998 tablet_service.cc:1511] Processing CreateTablet for tablet 2185927c34434da38aa9e7776e483b1b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:17.187438 29071 tablet_service.cc:1511] Processing CreateTablet for tablet 99a1616611dd41e49ec3ffc7eb3136a9 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:17.188781 29071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a1616611dd41e49ec3ffc7eb3136a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.181222 29073 tablet_service.cc:1511] Processing CreateTablet for tablet 2185927c34434da38aa9e7776e483b1b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:17.190320 29073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2185927c34434da38aa9e7776e483b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.191629 28994 tablet_service.cc:1511] Processing CreateTablet for tablet f97f68627cfa4585b418a3c090e44c14 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:17.188406 28995 tablet_service.cc:1511] Processing CreateTablet for tablet 3dcde0796c504aa0ad4a5e79df6a169d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:17.184978 28996 tablet_service.cc:1511] Processing CreateTablet for tablet 99a1616611dd41e49ec3ffc7eb3136a9 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:17.193135 29070 tablet_service.cc:1511] Processing CreateTablet for tablet 3dcde0796c504aa0ad4a5e79df6a169d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:17.194471 29070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcde0796c504aa0ad4a5e79df6a169d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.194980 28993 tablet_service.cc:1511] Processing CreateTablet for tablet ddba9f7f2a6843c480dac4ac4efa4931 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:17.197187 29069 tablet_service.cc:1511] Processing CreateTablet for tablet f97f68627cfa4585b418a3c090e44c14 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:17.198577 29069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97f68627cfa4585b418a3c090e44c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.198781 28992 tablet_service.cc:1511] Processing CreateTablet for tablet 86e87ad9e9a5439d890ebaf60f043eb6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:17.201347 29068 tablet_service.cc:1511] Processing CreateTablet for tablet ddba9f7f2a6843c480dac4ac4efa4931 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:17.202221 28991 tablet_service.cc:1511] Processing CreateTablet for tablet 49241be91ca743cbac56eb24218ff775 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:17.203856 29067 tablet_service.cc:1511] Processing CreateTablet for tablet 86e87ad9e9a5439d890ebaf60f043eb6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:17.206116 29066 tablet_service.cc:1511] Processing CreateTablet for tablet 49241be91ca743cbac56eb24218ff775 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:17.179301 29074 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:17.208588 28998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2185927c34434da38aa9e7776e483b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.212752 28994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97f68627cfa4585b418a3c090e44c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.213443 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddba9f7f2a6843c480dac4ac4efa4931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.214013 28991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49241be91ca743cbac56eb24218ff775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.214800 28997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e583be49bed64dd2996afcdcffa95120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.215586 28992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e87ad9e9a5439d890ebaf60f043eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.202607 29068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddba9f7f2a6843c480dac4ac4efa4931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.216365 28995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcde0796c504aa0ad4a5e79df6a169d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.216823 29067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e87ad9e9a5439d890ebaf60f043eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.217121 29000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7216d7dc49494630845e9eeb30dc6902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.220361 28996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a1616611dd41e49ec3ffc7eb3136a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.223413 29074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.228508 29066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49241be91ca743cbac56eb24218ff775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.273118 28995 tablet_service.cc:1511] Processing CreateTablet for tablet ab75774d33164f0b9c0269be22426768 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:17.280510 28991 tablet_service.cc:1511] Processing CreateTablet for tablet 5cac0f90a42148b5923aa84c7b7cf00b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:17.281394 28995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab75774d33164f0b9c0269be22426768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.282085 28991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cac0f90a42148b5923aa84c7b7cf00b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.286290 29144 tablet_service.cc:1511] Processing CreateTablet for tablet 3dcde0796c504aa0ad4a5e79df6a169d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:17.287671 29144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcde0796c504aa0ad4a5e79df6a169d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.290253 29143 tablet_service.cc:1511] Processing CreateTablet for tablet f97f68627cfa4585b418a3c090e44c14 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:17.291643 29143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97f68627cfa4585b418a3c090e44c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.291308 29141 tablet_service.cc:1511] Processing CreateTablet for tablet 86e87ad9e9a5439d890ebaf60f043eb6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:17.293004 29142 tablet_service.cc:1511] Processing CreateTablet for tablet ddba9f7f2a6843c480dac4ac4efa4931 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:17.293087 29146 tablet_service.cc:1511] Processing CreateTablet for tablet e583be49bed64dd2996afcdcffa95120 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:17.294435 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddba9f7f2a6843c480dac4ac4efa4931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.294760 29147 tablet_service.cc:1511] Processing CreateTablet for tablet 2185927c34434da38aa9e7776e483b1b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:17.296504 29145 tablet_service.cc:1511] Processing CreateTablet for tablet 99a1616611dd41e49ec3ffc7eb3136a9 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:17.297106 29149 tablet_service.cc:1511] Processing CreateTablet for tablet 7216d7dc49494630845e9eeb30dc6902 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:17.290271 29140 tablet_service.cc:1511] Processing CreateTablet for tablet 49241be91ca743cbac56eb24218ff775 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:17.300015 29148 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:17.304689 29147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2185927c34434da38aa9e7776e483b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.308657 29145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a1616611dd41e49ec3ffc7eb3136a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.309063 29000 tablet_service.cc:1511] Processing CreateTablet for tablet 828bc1c562f941c89bdcc43cf79e5ece (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:17.310380 29000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 828bc1c562f941c89bdcc43cf79e5ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.312678 29141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e87ad9e9a5439d890ebaf60f043eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.313354 29148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.316759 29140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49241be91ca743cbac56eb24218ff775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.319061 29146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e583be49bed64dd2996afcdcffa95120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.319468 29199 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:17.323796 29149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7216d7dc49494630845e9eeb30dc6902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.331354 28993 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:17.337430 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.373888 28996 tablet_service.cc:1511] Processing CreateTablet for tablet 21d163c72e704bd0bd37c74d96761663 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:17.375788 28996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d163c72e704bd0bd37c74d96761663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.379995 28998 tablet_service.cc:1511] Processing CreateTablet for tablet 3aa255ba50b343eca7717515ec37d959 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:17.364405 28992 tablet_service.cc:1511] Processing CreateTablet for tablet b39693173a844463b5f0947904461289 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:17.398437 29199 tablet_bootstrap.cc:654] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.398564 29147 tablet_service.cc:1511] Processing CreateTablet for tablet ab75774d33164f0b9c0269be22426768 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:17.400844 29075 tablet_service.cc:1511] Processing CreateTablet for tablet ab75774d33164f0b9c0269be22426768 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:17.416419 29200 tablet_bootstrap.cc:492] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:17.416749 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 5cac0f90a42148b5923aa84c7b7cf00b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:17.418062 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cac0f90a42148b5923aa84c7b7cf00b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.419479 29070 tablet_service.cc:1511] Processing CreateTablet for tablet 828bc1c562f941c89bdcc43cf79e5ece (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:17.420714 29147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab75774d33164f0b9c0269be22426768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.420994 29070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 828bc1c562f941c89bdcc43cf79e5ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.421285 29072 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:17.422526 29072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.428071 29067 tablet_service.cc:1511] Processing CreateTablet for tablet 21d163c72e704bd0bd37c74d96761663 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:17.429632 29067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d163c72e704bd0bd37c74d96761663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.432822 29073 tablet_service.cc:1511] Processing CreateTablet for tablet b39693173a844463b5f0947904461289 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:17.434477 29073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b39693173a844463b5f0947904461289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.438035 29066 tablet_service.cc:1511] Processing CreateTablet for tablet 5cac0f90a42148b5923aa84c7b7cf00b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:17.439800 29066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cac0f90a42148b5923aa84c7b7cf00b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.442025 28998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aa255ba50b343eca7717515ec37d959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.445566 28995 tablet_service.cc:1511] Processing CreateTablet for tablet 2bbf1db4b075436695fd521f5a5b2e9d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:17.454210 29068 tablet_service.cc:1511] Processing CreateTablet for tablet 3aa255ba50b343eca7717515ec37d959 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:17.465830 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 828bc1c562f941c89bdcc43cf79e5ece (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:17.476462 29201 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:17.476336 29071 tablet_service.cc:1511] Processing CreateTablet for tablet 29895dd2e2b6460b8106269935db24f7 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:17.480171 28992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b39693173a844463b5f0947904461289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.481806 29075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab75774d33164f0b9c0269be22426768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.484206 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 828bc1c562f941c89bdcc43cf79e5ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.489490 29069 tablet_service.cc:1511] Processing CreateTablet for tablet e21ca94fafcc4923a671c355b710f10d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:17.491305 28999 tablet_service.cc:1511] Processing CreateTablet for tablet f68cf8a7e5894c679a8bba8d7d2374a5 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:17.495944 29200 tablet_bootstrap.cc:654] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.508522 28994 tablet_service.cc:1511] Processing CreateTablet for tablet 29895dd2e2b6460b8106269935db24f7 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:17.509836 28995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bbf1db4b075436695fd521f5a5b2e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.510171 29068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aa255ba50b343eca7717515ec37d959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.513309 29140 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:17.502360 28993 tablet_service.cc:1511] Processing CreateTablet for tablet e21ca94fafcc4923a671c355b710f10d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:17.514562 29140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.465881 29070 tablet_service.cc:1511] Processing CreateTablet for tablet 2bbf1db4b075436695fd521f5a5b2e9d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:17.516249 29000 tablet_service.cc:1511] Processing CreateTablet for tablet eb070bde5f3744eb90ee5cf3dec8fcd4 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:17.517764 29201 tablet_bootstrap.cc:654] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.517869 29149 tablet_service.cc:1511] Processing CreateTablet for tablet 21d163c72e704bd0bd37c74d96761663 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:17.519145 29149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d163c72e704bd0bd37c74d96761663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.521775 28991 tablet_service.cc:1511] Processing CreateTablet for tablet d83f8e6d06054712b0c7cd2c5690ec63 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:17.523029 28991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d83f8e6d06054712b0c7cd2c5690ec63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.526057 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e21ca94fafcc4923a671c355b710f10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.526283 28997 tablet_service.cc:1511] Processing CreateTablet for tablet e225d0308db94fb9ab850a0b06a63551 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:17.529217 29146 tablet_service.cc:1511] Processing CreateTablet for tablet b39693173a844463b5f0947904461289 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:17.530480 29146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b39693173a844463b5f0947904461289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.532364 28999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68cf8a7e5894c679a8bba8d7d2374a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.534426 28994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29895dd2e2b6460b8106269935db24f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.536361 29000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb070bde5f3744eb90ee5cf3dec8fcd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.549021 29143 tablet_service.cc:1511] Processing CreateTablet for tablet e21ca94fafcc4923a671c355b710f10d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:17.550379 29143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e21ca94fafcc4923a671c355b710f10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.551788 29069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e21ca94fafcc4923a671c355b710f10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.553251 29074 tablet_service.cc:1511] Processing CreateTablet for tablet eb070bde5f3744eb90ee5cf3dec8fcd4 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:17.554574 29074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb070bde5f3744eb90ee5cf3dec8fcd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.556458 29070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bbf1db4b075436695fd521f5a5b2e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.558228 29148 tablet_service.cc:1511] Processing CreateTablet for tablet eb070bde5f3744eb90ee5cf3dec8fcd4 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:17.559497 29148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb070bde5f3744eb90ee5cf3dec8fcd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.556483 29141 tablet_service.cc:1511] Processing CreateTablet for tablet 29895dd2e2b6460b8106269935db24f7 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:17.561360 29141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29895dd2e2b6460b8106269935db24f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.571906 28996 tablet_service.cc:1511] Processing CreateTablet for tablet 26c8dceb42c541c8906c424b9bc9bb17 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:17.573838 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 3aa255ba50b343eca7717515ec37d959 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:17.575074 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aa255ba50b343eca7717515ec37d959. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:17.578963 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:17.579676 29071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29895dd2e2b6460b8106269935db24f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.549238 29147 tablet_service.cc:1511] Processing CreateTablet for tablet 2bbf1db4b075436695fd521f5a5b2e9d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:17.582737 29075 tablet_service.cc:1511] Processing CreateTablet for tablet f68cf8a7e5894c679a8bba8d7d2374a5 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:17.592104 29147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bbf1db4b075436695fd521f5a5b2e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.594390 28994 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb3976ea75a4cb3b4f543d842c2409c (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:17.596946 29067 tablet_service.cc:1511] Processing CreateTablet for tablet d83f8e6d06054712b0c7cd2c5690ec63 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:17.599781 29072 tablet_service.cc:1511] Processing CreateTablet for tablet e225d0308db94fb9ab850a0b06a63551 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:17.602411 28992 tablet_service.cc:1511] Processing CreateTablet for tablet 5ec27c4bcd6b4eb3a5165f616c4ad641 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:17.584794 29075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68cf8a7e5894c679a8bba8d7d2374a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.609215 29067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d83f8e6d06054712b0c7cd2c5690ec63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.610617 29072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e225d0308db94fb9ab850a0b06a63551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.615351 29000 tablet_service.cc:1511] Processing CreateTablet for tablet 3eaf29b1a5c049b1bc9159a53ce52afb (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:17.622638 29201 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:17.623229 29201 ts_tablet_manager.cc:1403] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.147s	user 0.005s	sys 0.008s
I20260709 13:10:17.626353 29201 raft_consensus.cc:359] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:17.627389 29201 raft_consensus.cc:385] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:17.627805 29201 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:17.628720 29201 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:17.633291 28997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e225d0308db94fb9ab850a0b06a63551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.634289 29199 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:17.634654 29200 tablet_bootstrap.cc:492] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:17.634812 29199 ts_tablet_manager.cc:1403] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.316s	user 0.017s	sys 0.009s
I20260709 13:10:17.635130 29200 ts_tablet_manager.cc:1403] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.219s	user 0.007s	sys 0.004s
I20260709 13:10:17.638000 29199 raft_consensus.cc:359] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:17.638245 29200 raft_consensus.cc:359] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:17.638785 29199 raft_consensus.cc:385] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:17.638975 29200 raft_consensus.cc:385] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:17.639317 29200 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:17.640125 28994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb3976ea75a4cb3b4f543d842c2409c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.630267 28993 tablet_service.cc:1511] Processing CreateTablet for tablet 0df5a8a9dbd84e3bad34bf60478e7c98 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:17.640410 29200 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:17.639289 29199 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:17.644238 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df5a8a9dbd84e3bad34bf60478e7c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.644727 29199 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:17.658425 29140 tablet_service.cc:1511] Processing CreateTablet for tablet f68cf8a7e5894c679a8bba8d7d2374a5 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:17.659811 29140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68cf8a7e5894c679a8bba8d7d2374a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.662803 29141 tablet_service.cc:1511] Processing CreateTablet for tablet d83f8e6d06054712b0c7cd2c5690ec63 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:17.664122 29141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d83f8e6d06054712b0c7cd2c5690ec63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.675159 28998 tablet_service.cc:1511] Processing CreateTablet for tablet ea8e44f6f34a4291baf426298bc260e6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:17.681737 29144 tablet_service.cc:1511] Processing CreateTablet for tablet 26c8dceb42c541c8906c424b9bc9bb17 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:17.683218 29144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c8dceb42c541c8906c424b9bc9bb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.660149 28992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec27c4bcd6b4eb3a5165f616c4ad641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.671453 29142 tablet_service.cc:1511] Processing CreateTablet for tablet e225d0308db94fb9ab850a0b06a63551 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:17.716459 29184 heartbeater.cc:499] Master 127.27.63.126:43467 was elected leader, sending a full tablet report...
I20260709 13:10:17.717741 29146 tablet_service.cc:1511] Processing CreateTablet for tablet 3eaf29b1a5c049b1bc9159a53ce52afb (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:17.725319 28999 tablet_service.cc:1511] Processing CreateTablet for tablet 1db879e214974ab38bc94a40afcf6e1e (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:17.725661 29148 tablet_service.cc:1511] Processing CreateTablet for tablet 0df5a8a9dbd84e3bad34bf60478e7c98 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:17.727550 29200 ts_tablet_manager.cc:1434] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.092s	user 0.024s	sys 0.020s
I20260709 13:10:17.728876 29199 ts_tablet_manager.cc:1434] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.020s
I20260709 13:10:17.728965 28995 tablet_service.cc:1511] Processing CreateTablet for tablet 640d00aba1b541e4abb90c63e6b8720d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:17.729643 29035 heartbeater.cc:499] Master 127.27.63.126:43467 was elected leader, sending a full tablet report...
I20260709 13:10:17.729946 29199 tablet_bootstrap.cc:492] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:17.730626 29110 heartbeater.cc:499] Master 127.27.63.126:43467 was elected leader, sending a full tablet report...
I20260709 13:10:17.736991 29145 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb3976ea75a4cb3b4f543d842c2409c (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:17.738651 29075 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb3976ea75a4cb3b4f543d842c2409c (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:17.739894 29075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb3976ea75a4cb3b4f543d842c2409c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.744163 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e225d0308db94fb9ab850a0b06a63551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.744486 29200 tablet_bootstrap.cc:492] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:17.712548 28996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c8dceb42c541c8906c424b9bc9bb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.749522 29068 tablet_service.cc:1511] Processing CreateTablet for tablet 3eaf29b1a5c049b1bc9159a53ce52afb (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:17.750859 29068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eaf29b1a5c049b1bc9159a53ce52afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.751813 28998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea8e44f6f34a4291baf426298bc260e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.752734 29145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb3976ea75a4cb3b4f543d842c2409c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.755398 29067 tablet_service.cc:1511] Processing CreateTablet for tablet 0df5a8a9dbd84e3bad34bf60478e7c98 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:17.756337 29000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eaf29b1a5c049b1bc9159a53ce52afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.756723 29146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eaf29b1a5c049b1bc9159a53ce52afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.757673 29148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df5a8a9dbd84e3bad34bf60478e7c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.757795 28999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1db879e214974ab38bc94a40afcf6e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.758558 29201 ts_tablet_manager.cc:1434] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.135s	user 0.010s	sys 0.008s
I20260709 13:10:17.759662 29201 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:17.761752 28995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 640d00aba1b541e4abb90c63e6b8720d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.764062 28991 tablet_service.cc:1511] Processing CreateTablet for tablet fa4501421f5146e4a7b79fd8697f8f57 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:17.765304 28991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa4501421f5146e4a7b79fd8697f8f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.763674 29071 tablet_service.cc:1511] Processing CreateTablet for tablet ea8e44f6f34a4291baf426298bc260e6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:17.773130 29071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea8e44f6f34a4291baf426298bc260e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.735654 29073 tablet_service.cc:1511] Processing CreateTablet for tablet 26c8dceb42c541c8906c424b9bc9bb17 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:17.766156 29066 tablet_service.cc:1511] Processing CreateTablet for tablet 640d00aba1b541e4abb90c63e6b8720d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:17.752161 29074 tablet_service.cc:1511] Processing CreateTablet for tablet 5ec27c4bcd6b4eb3a5165f616c4ad641 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:17.792841 29149 tablet_service.cc:1511] Processing CreateTablet for tablet 5ec27c4bcd6b4eb3a5165f616c4ad641 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:17.798564 29201 tablet_bootstrap.cc:654] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.803017 28996 tablet_service.cc:1511] Processing CreateTablet for tablet 342b8b8f6e7742c897bcd3495fecb00a (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:17.804384 28996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342b8b8f6e7742c897bcd3495fecb00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.811069 29199 tablet_bootstrap.cc:654] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.817271 28993 tablet_service.cc:1511] Processing CreateTablet for tablet 94045d5a8709418e82919c9fbf804582 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:17.818231 29200 tablet_bootstrap.cc:654] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:17.824719 29149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec27c4bcd6b4eb3a5165f616c4ad641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.843534 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94045d5a8709418e82919c9fbf804582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.828570 29067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df5a8a9dbd84e3bad34bf60478e7c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.829221 29206 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.848212 29206 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:17.848615 29074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec27c4bcd6b4eb3a5165f616c4ad641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.850045 29066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 640d00aba1b541e4abb90c63e6b8720d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.850695 29073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c8dceb42c541c8906c424b9bc9bb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.833639 29068 tablet_service.cc:1511] Processing CreateTablet for tablet 1db879e214974ab38bc94a40afcf6e1e (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:17.836705 29069 tablet_service.cc:1511] Processing CreateTablet for tablet ce7f7e5792294b1f99a1bf4ee332a76c (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:17.837623 29072 tablet_service.cc:1511] Processing CreateTablet for tablet fa4501421f5146e4a7b79fd8697f8f57 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:17.843459 29070 tablet_service.cc:1511] Processing CreateTablet for tablet 94045d5a8709418e82919c9fbf804582 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:17.826438 29141 tablet_service.cc:1511] Processing CreateTablet for tablet fa4501421f5146e4a7b79fd8697f8f57 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:17.870271 29140 tablet_service.cc:1511] Processing CreateTablet for tablet ea8e44f6f34a4291baf426298bc260e6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:17.798707 29000 tablet_service.cc:1511] Processing CreateTablet for tablet ce7f7e5792294b1f99a1bf4ee332a76c (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:17.850867 29206 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:17.839565 29075 tablet_service.cc:1511] Processing CreateTablet for tablet 342b8b8f6e7742c897bcd3495fecb00a (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:17.883196 29000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce7f7e5792294b1f99a1bf4ee332a76c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.839956 29143 tablet_service.cc:1511] Processing CreateTablet for tablet 640d00aba1b541e4abb90c63e6b8720d (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:17.884469 29068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1db879e214974ab38bc94a40afcf6e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.886436 28992 tablet_service.cc:1511] Processing CreateTablet for tablet 2bf52cc065f943c9a83088bbeea5c794 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:17.887229 29141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa4501421f5146e4a7b79fd8697f8f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.912618 29143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 640d00aba1b541e4abb90c63e6b8720d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.917052 29147 tablet_service.cc:1511] Processing CreateTablet for tablet 342b8b8f6e7742c897bcd3495fecb00a (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:17.917528 28994 tablet_service.cc:1511] Processing CreateTablet for tablet bf1e3918423546bcafc9359aca2ca811 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:17.918401 29147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342b8b8f6e7742c897bcd3495fecb00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.920486 29072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa4501421f5146e4a7b79fd8697f8f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.924458 29069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce7f7e5792294b1f99a1bf4ee332a76c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:17.878808 28895 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46642us
I20260709 13:10:17.927384 29070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94045d5a8709418e82919c9fbf804582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.928256 29140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea8e44f6f34a4291baf426298bc260e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.928887 28994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf1e3918423546bcafc9359aca2ca811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.928745 28997 tablet_service.cc:1511] Processing CreateTablet for tablet dafcee70c76341f2a36a461c97f2b3ba (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 13:10:17.930049 28894 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 71182.3us
I20260709 13:10:17.937250 29145 tablet_service.cc:1511] Processing CreateTablet for tablet ce7f7e5792294b1f99a1bf4ee332a76c (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:17.938596 29145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce7f7e5792294b1f99a1bf4ee332a76c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.895241 29146 tablet_service.cc:1511] Processing CreateTablet for tablet 1db879e214974ab38bc94a40afcf6e1e (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:17.945184 29146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1db879e214974ab38bc94a40afcf6e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.951231 28997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafcee70c76341f2a36a461c97f2b3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.958423 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 94045d5a8709418e82919c9fbf804582 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:17.959743 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94045d5a8709418e82919c9fbf804582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.960796 28992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf52cc065f943c9a83088bbeea5c794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.963718 29148 tablet_service.cc:1511] Processing CreateTablet for tablet 68c287f2818e4a23b258f6c19741c26a (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:17.974440 28998 tablet_service.cc:1511] Processing CreateTablet for tablet 8da4f9ce63d54a63a8b9de11ba485699 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:17.986183 29148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68c287f2818e4a23b258f6c19741c26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.989096 29141 tablet_service.cc:1511] Processing CreateTablet for tablet bf1e3918423546bcafc9359aca2ca811 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:17.990576 29141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf1e3918423546bcafc9359aca2ca811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:17.992919 28995 tablet_service.cc:1511] Processing CreateTablet for tablet 00c2e050cffc43618bffbaaa661fcfa6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:17.994445 29207 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.994971 29207 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:17.998018 29199 tablet_bootstrap.cc:492] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.000933 28999 tablet_service.cc:1511] Processing CreateTablet for tablet 68c287f2818e4a23b258f6c19741c26a (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:17.998541 29199 ts_tablet_manager.cc:1403] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.269s	user 0.003s	sys 0.028s
I20260709 13:10:18.007387 29075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342b8b8f6e7742c897bcd3495fecb00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.019168 29144 tablet_service.cc:1511] Processing CreateTablet for tablet 2bf52cc065f943c9a83088bbeea5c794 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:18.021317 29200 tablet_bootstrap.cc:492] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.021843 29200 ts_tablet_manager.cc:1403] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.280s	user 0.004s	sys 0.013s
I20260709 13:10:18.036378 29201 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.037091 29201 ts_tablet_manager.cc:1403] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.278s	user 0.019s	sys 0.028s
I20260709 13:10:18.039057 29205 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.039645 29205 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.000118 29207 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.054899 29205 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:18.056401 29144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf52cc065f943c9a83088bbeea5c794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.060757 29200 raft_consensus.cc:359] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.061525 29200 raft_consensus.cc:385] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.061834 29200 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.063566 29066 tablet_service.cc:1511] Processing CreateTablet for tablet 68c287f2818e4a23b258f6c19741c26a (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:18.064960 29066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68c287f2818e4a23b258f6c19741c26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.045966 28998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da4f9ce63d54a63a8b9de11ba485699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.062569 29200 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.075706 29200 ts_tablet_manager.cc:1434] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.054s	user 0.000s	sys 0.007s
I20260709 13:10:18.062045 29140 tablet_service.cc:1511] Processing CreateTablet for tablet dafcee70c76341f2a36a461c97f2b3ba (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:18.086783 29140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafcee70c76341f2a36a461c97f2b3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.085199 29200 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:18.093642 29069 tablet_service.cc:1511] Processing CreateTablet for tablet bf1e3918423546bcafc9359aca2ca811 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:18.095309 29069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf1e3918423546bcafc9359aca2ca811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.096796 28993 tablet_service.cc:1511] Processing CreateTablet for tablet d012d02d5bf34cbf8f4451c054b36f63 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:18.101998 29200 tablet_bootstrap.cc:654] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.106705 29149 tablet_service.cc:1511] Processing CreateTablet for tablet 8da4f9ce63d54a63a8b9de11ba485699 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:18.108646 29201 raft_consensus.cc:359] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.121004 29201 raft_consensus.cc:385] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.146557 29199 raft_consensus.cc:359] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.147364 29199 raft_consensus.cc:385] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.147657 29199 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.134658 28995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c2e050cffc43618bffbaaa661fcfa6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:18.122337 28895 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43657.4us
I20260709 13:10:18.140830 29201 raft_consensus.cc:740] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.174829 28999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68c287f2818e4a23b258f6c19741c26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.175622 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d012d02d5bf34cbf8f4451c054b36f63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.186661 29073 tablet_service.cc:1511] Processing CreateTablet for tablet 2bf52cc065f943c9a83088bbeea5c794 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:18.193588 29072 tablet_service.cc:1511] Processing CreateTablet for tablet dafcee70c76341f2a36a461c97f2b3ba (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:18.165839 29199 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.196883 29000 tablet_service.cc:1511] Processing CreateTablet for tablet 31c5d53651b5426780911903843f0bc3 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:18.197207 28991 tablet_service.cc:1511] Processing CreateTablet for tablet 00715172c7364d849f262c553c53e5fe (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:18.197750 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:18.199821 29199 ts_tablet_manager.cc:1434] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.180s	user 0.007s	sys 0.000s
I20260709 13:10:18.198987 28996 tablet_service.cc:1511] Processing CreateTablet for tablet 2ed915f0be134d71b36d8fc8f7b5830b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:18.200834 29199 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
W20260709 13:10:18.200729 29048 leader_election.cc:343] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.201128 29201 consensus_queue.cc:260] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.203408 29201 ts_tablet_manager.cc:1434] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.166s	user 0.005s	sys 0.002s
I20260709 13:10:18.208528 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:18.196919 28995 tablet_service.cc:1511] Processing CreateTablet for tablet d8340da85294490fb88f62f80393531f (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
W20260709 13:10:18.211027 28976 leader_election.cc:343] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.198335 29000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31c5d53651b5426780911903843f0bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.214303 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:18.214996 28991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00715172c7364d849f262c553c53e5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.214989 29010 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:18.216939 29067 tablet_service.cc:1511] Processing CreateTablet for tablet 00c2e050cffc43618bffbaaa661fcfa6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:18.218362 28995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8340da85294490fb88f62f80393531f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.216962 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:18.221009 29205 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.170368 29149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da4f9ce63d54a63a8b9de11ba485699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.221401 29205 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.221781 29205 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.187953 29073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf52cc065f943c9a83088bbeea5c794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.220702 29074 tablet_service.cc:1511] Processing CreateTablet for tablet 8da4f9ce63d54a63a8b9de11ba485699 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:18.231707 28996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ed915f0be134d71b36d8fc8f7b5830b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.235051 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 00c2e050cffc43618bffbaaa661fcfa6 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:18.236495 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:18.237886 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c2e050cffc43618bffbaaa661fcfa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.243263 29072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafcee70c76341f2a36a461c97f2b3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.251885 29074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da4f9ce63d54a63a8b9de11ba485699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.253461 29200 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.253966 29200 ts_tablet_manager.cc:1403] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.177s	user 0.027s	sys 0.020s
I20260709 13:10:18.257182 29200 raft_consensus.cc:359] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.257929 29200 raft_consensus.cc:385] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.258247 29200 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.262799 29071 tablet_service.cc:1511] Processing CreateTablet for tablet d012d02d5bf34cbf8f4451c054b36f63 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:18.263823 29147 tablet_service.cc:1511] Processing CreateTablet for tablet 2ed915f0be134d71b36d8fc8f7b5830b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:18.265092 29147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ed915f0be134d71b36d8fc8f7b5830b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.265744 29146 tablet_service.cc:1511] Processing CreateTablet for tablet 31c5d53651b5426780911903843f0bc3 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:18.266518 29140 tablet_service.cc:1511] Processing CreateTablet for tablet d8340da85294490fb88f62f80393531f (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:18.267031 29146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31c5d53651b5426780911903843f0bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.267335 29201 tablet_bootstrap.cc:492] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.271922 29205 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.273239 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:18.274027 29159 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:18.274899 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:18.275269 28975 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:18.275518 29010 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.277642 29200 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.266251 28998 tablet_service.cc:1511] Processing CreateTablet for tablet 824fc688db264856b025a29c651eec78 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:18.283633 29010 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:18.303016 29207 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.274056 29205 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:18.303465 29068 tablet_service.cc:1511] Processing CreateTablet for tablet 00715172c7364d849f262c553c53e5fe (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:18.307411 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:18.313156 29069 tablet_service.cc:1511] Processing CreateTablet for tablet d8340da85294490fb88f62f80393531f (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:18.317252 29201 tablet_bootstrap.cc:654] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.315999 29143 tablet_service.cc:1511] Processing CreateTablet for tablet 00715172c7364d849f262c553c53e5fe (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:18.283778 29070 tablet_service.cc:1511] Processing CreateTablet for tablet 31c5d53651b5426780911903843f0bc3 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:18.288373 29140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8340da85294490fb88f62f80393531f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.292279 29145 tablet_service.cc:1511] Processing CreateTablet for tablet d012d02d5bf34cbf8f4451c054b36f63 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:18.317903 29205 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.295490 28998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824fc688db264856b025a29c651eec78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.300307 29067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00c2e050cffc43618bffbaaa661fcfa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.300366 29200 ts_tablet_manager.cc:1434] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 13:10:18.333002 29200 tablet_bootstrap.cc:492] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
W20260709 13:10:18.302033 29050 leader_election.cc:343] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.320833 29145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d012d02d5bf34cbf8f4451c054b36f63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.307878 29075 tablet_service.cc:1511] Processing CreateTablet for tablet 2ed915f0be134d71b36d8fc8f7b5830b (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:18.333906 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:18.334807 29206 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.343705 29201 tablet_bootstrap.cc:492] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.344914 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:18.345127 29201 ts_tablet_manager.cc:1403] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.114s	user 0.008s	sys 0.004s
I20260709 13:10:18.326262 29066 tablet_service.cc:1511] Processing CreateTablet for tablet 824fc688db264856b025a29c651eec78 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:18.346661 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:18.329003 28994 tablet_service.cc:1511] Processing CreateTablet for tablet e1a1cae5d4eb46b288c03a3675d24209 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:18.348507 28994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a1cae5d4eb46b288c03a3675d24209. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:18.351717 29125 leader_election.cc:343] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.329304 29070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31c5d53651b5426780911903843f0bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.352734 29068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00715172c7364d849f262c553c53e5fe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:18.353148 29125 leader_election.cc:343] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.294585 29144 tablet_service.cc:1511] Processing CreateTablet for tablet 824fc688db264856b025a29c651eec78 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:18.354712 29201 raft_consensus.cc:359] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.355461 29201 raft_consensus.cc:385] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.355752 29201 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.322631 29142 tablet_service.cc:1511] Processing CreateTablet for tablet e1a1cae5d4eb46b288c03a3675d24209 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:18.356351 29071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d012d02d5bf34cbf8f4451c054b36f63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.358217 29069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8340da85294490fb88f62f80393531f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.362772 29072 tablet_service.cc:1511] Processing CreateTablet for tablet e1a1cae5d4eb46b288c03a3675d24209 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:18.363413 29206 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.364126 29072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a1cae5d4eb46b288c03a3675d24209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.363842 29206 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.365923 29206 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.366793 29075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ed915f0be134d71b36d8fc8f7b5830b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.367115 29199 tablet_bootstrap.cc:654] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.367635 29201 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.373354 29200 tablet_bootstrap.cc:654] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.375949 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:18.323700 28993 tablet_service.cc:1511] Processing CreateTablet for tablet 6a08e186ea0749ae979329103591a933 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:10:18.380865 29048 leader_election.cc:343] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.381464 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a08e186ea0749ae979329103591a933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.376262 29143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00715172c7364d849f262c553c53e5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.384208 29066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824fc688db264856b025a29c651eec78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.389653 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a1cae5d4eb46b288c03a3675d24209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.393712 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:18.395272 29050 leader_election.cc:343] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.395722 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:18.398892 29206 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.404068 29201 ts_tablet_manager.cc:1434] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.057s	user 0.011s	sys 0.001s
I20260709 13:10:18.404249 29073 tablet_service.cc:1511] Processing CreateTablet for tablet 6a08e186ea0749ae979329103591a933 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:18.405109 29201 tablet_bootstrap.cc:492] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.405535 29073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a08e186ea0749ae979329103591a933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.401976 29144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824fc688db264856b025a29c651eec78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.424770 28996 tablet_service.cc:1511] Processing CreateTablet for tablet c141925650a74de58026be545d874618 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:18.426628 28996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c141925650a74de58026be545d874618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.437727 29201 tablet_bootstrap.cc:654] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.444262 29066 tablet_service.cc:1511] Processing CreateTablet for tablet c141925650a74de58026be545d874618 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:18.446189 29066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c141925650a74de58026be545d874618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.454027 28999 tablet_service.cc:1511] Processing CreateTablet for tablet 027a3f0877db4e259d94a8e296822644 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:18.455833 28999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027a3f0877db4e259d94a8e296822644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.465503 29206 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.466065 29206 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.467868 29199 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.468397 29199 ts_tablet_manager.cc:1403] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.268s	user 0.016s	sys 0.007s
I20260709 13:10:18.468356 29206 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.469455 29147 tablet_service.cc:1511] Processing CreateTablet for tablet 6a08e186ea0749ae979329103591a933 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:18.470594 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:18.471314 29225 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.471771 29225 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:18.472271 29050 leader_election.cc:343] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.473510 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:18.475489 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:18.476836 29125 leader_election.cc:343] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.477187 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:18.479272 29122 leader_election.cc:343] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.479843 29122 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:18.469547 29200 tablet_bootstrap.cc:492] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.481966 29200 ts_tablet_manager.cc:1403] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.149s	user 0.013s	sys 0.019s
I20260709 13:10:18.485311 29225 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:18.486367 29225 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.489279 29145 tablet_service.cc:1511] Processing CreateTablet for tablet c141925650a74de58026be545d874618 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:18.490595 29145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c141925650a74de58026be545d874618. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:18.490487 29048 leader_election.cc:343] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.491003 29048 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:18.492151 29206 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.497516 29199 raft_consensus.cc:359] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.498276 29199 raft_consensus.cc:385] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.498569 29199 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.497642 29200 raft_consensus.cc:359] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.507972 29068 tablet_service.cc:1511] Processing CreateTablet for tablet 027a3f0877db4e259d94a8e296822644 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:18.509795 29068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027a3f0877db4e259d94a8e296822644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.499274 29199 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.502694 29200 raft_consensus.cc:385] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.519940 29147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a08e186ea0749ae979329103591a933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.520087 29200 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.521088 29200 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.521507 29207 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.522063 29207 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.523066 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 027a3f0877db4e259d94a8e296822644 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:18.524211 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027a3f0877db4e259d94a8e296822644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.517549 29205 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:18.527189 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:18.524111 29207 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.527146 29205 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.530094 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:18.531215 28976 leader_election.cc:343] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.531147 28991 tablet_service.cc:1511] Processing CreateTablet for tablet 8d989a55ed204649a0440142ce55e5ef (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20260709 13:10:18.531738 28975 leader_election.cc:343] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.532256 28975 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:18.532562 28991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d989a55ed204649a0440142ce55e5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.533334 29207 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.523470 29200 ts_tablet_manager.cc:1434] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:10:18.525034 29199 ts_tablet_manager.cc:1434] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.055s	user 0.005s	sys 0.002s
I20260709 13:10:18.552575 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 8d989a55ed204649a0440142ce55e5ef (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:18.554209 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d989a55ed204649a0440142ce55e5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.554015 29074 tablet_service.cc:1511] Processing CreateTablet for tablet 8d989a55ed204649a0440142ce55e5ef (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:18.555392 29074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d989a55ed204649a0440142ce55e5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.568625 29199 tablet_bootstrap.cc:492] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:18.575556 29205 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.576115 29205 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.576485 29075 tablet_service.cc:1511] Processing CreateTablet for tablet 57dd7e7308fc43e0b0983412129aa471 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:18.577793 29075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dd7e7308fc43e0b0983412129aa471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.578999 29205 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:18.579123 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:18.579577 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:18.580530 29122 leader_election.cc:343] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:18.581971 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:18.582640 28906 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:18.588142 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 57dd7e7308fc43e0b0983412129aa471 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:18.588905 28996 tablet_service.cc:1511] Processing CreateTablet for tablet 57dd7e7308fc43e0b0983412129aa471 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:18.591259 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dd7e7308fc43e0b0983412129aa471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.591382 28996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dd7e7308fc43e0b0983412129aa471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.596421 28998 tablet_service.cc:1511] Processing CreateTablet for tablet 9b9b21ce33904ed489b959d5aef0349f (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:18.597808 28998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b9b21ce33904ed489b959d5aef0349f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.600327 29200 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
W20260709 13:10:18.602514 29125 leader_election.cc:343] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.602991 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:18.590320 29201 tablet_bootstrap.cc:492] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.609565 29146 tablet_service.cc:1511] Processing CreateTablet for tablet 9b9b21ce33904ed489b959d5aef0349f (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:18.611091 29146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b9b21ce33904ed489b959d5aef0349f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.611804 29201 ts_tablet_manager.cc:1403] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.207s	user 0.021s	sys 0.022s
I20260709 13:10:18.613113 28993 tablet_service.cc:1511] Processing CreateTablet for tablet 72d50ab8517242a1bb42501ec563c433 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:18.614496 29200 tablet_bootstrap.cc:654] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.614827 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d50ab8517242a1bb42501ec563c433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.617287 29073 tablet_service.cc:1511] Processing CreateTablet for tablet 72d50ab8517242a1bb42501ec563c433 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:18.618530 29073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d50ab8517242a1bb42501ec563c433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.618368 29201 raft_consensus.cc:359] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.619118 29201 raft_consensus.cc:385] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.619444 29201 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.620134 29201 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.622385 29201 ts_tablet_manager.cc:1434] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:10:18.623365 29201 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.624119 29205 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.627571 29068 tablet_service.cc:1511] Processing CreateTablet for tablet 9b9b21ce33904ed489b959d5aef0349f (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:18.644362 28991 tablet_service.cc:1511] Processing CreateTablet for tablet 44c8fda62bd34c2f87eb2114b065741e (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:18.645645 28991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c8fda62bd34c2f87eb2114b065741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.645792 29066 tablet_service.cc:1511] Processing CreateTablet for tablet 44c8fda62bd34c2f87eb2114b065741e (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:18.647034 29066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c8fda62bd34c2f87eb2114b065741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.653622 29200 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.654208 29200 ts_tablet_manager.cc:1403] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.081s	user 0.015s	sys 0.003s
I20260709 13:10:18.655722 29068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b9b21ce33904ed489b959d5aef0349f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.658447 29200 raft_consensus.cc:359] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.663105 29200 raft_consensus.cc:385] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.662266 29145 tablet_service.cc:1511] Processing CreateTablet for tablet 44c8fda62bd34c2f87eb2114b065741e (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:18.667320 29145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c8fda62bd34c2f87eb2114b065741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.675336 29200 raft_consensus.cc:740] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.676177 29200 consensus_queue.cc:260] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.679005 28993 tablet_service.cc:1511] Processing CreateTablet for tablet 6415e058581b4b7eb883c6e96b9fbfcf (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:18.680189 28993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6415e058581b4b7eb883c6e96b9fbfcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.682200 29143 tablet_service.cc:1511] Processing CreateTablet for tablet 6415e058581b4b7eb883c6e96b9fbfcf (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:18.683471 29143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6415e058581b4b7eb883c6e96b9fbfcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.686178 28991 tablet_service.cc:1511] Processing CreateTablet for tablet c32aba72953748b4bb7a95efc76501ff (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:18.687229 29199 tablet_bootstrap.cc:654] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.687430 28991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c32aba72953748b4bb7a95efc76501ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.689113 29142 tablet_service.cc:1511] Processing CreateTablet for tablet 72d50ab8517242a1bb42501ec563c433 (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:18.690189 29142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d50ab8517242a1bb42501ec563c433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.692391 29067 tablet_service.cc:1511] Processing CreateTablet for tablet c32aba72953748b4bb7a95efc76501ff (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:18.693585 29067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c32aba72953748b4bb7a95efc76501ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.698671 29144 tablet_service.cc:1511] Processing CreateTablet for tablet c32aba72953748b4bb7a95efc76501ff (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:18.699644 29201 tablet_bootstrap.cc:654] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.699919 29144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c32aba72953748b4bb7a95efc76501ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.705965 29073 tablet_service.cc:1511] Processing CreateTablet for tablet 6415e058581b4b7eb883c6e96b9fbfcf (DEFAULT_TABLE table=test-workload [id=db7088a4cae2474f8ae2dd254e96e144]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:18.707023 29200 ts_tablet_manager.cc:1434] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.001s
I20260709 13:10:18.712944 29073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6415e058581b4b7eb883c6e96b9fbfcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:18.728433 29200 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:18.738737 29206 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.739241 29206 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.740976 29206 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.742138 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:18.743194 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:18.743690 29050 leader_election.cc:343] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:18.745548 29048 leader_election.cc:343] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.745985 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:18.744256 29200 tablet_bootstrap.cc:654] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.748093 29206 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.754176 29201 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.754726 29201 ts_tablet_manager.cc:1403] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.132s	user 0.013s	sys 0.002s
I20260709 13:10:18.755317 29199 tablet_bootstrap.cc:492] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.755842 29199 ts_tablet_manager.cc:1403] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.187s	user 0.013s	sys 0.001s
I20260709 13:10:18.759590 29201 raft_consensus.cc:359] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.760639 29201 raft_consensus.cc:385] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.761042 29201 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.761654 29199 raft_consensus.cc:359] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.762291 29199 raft_consensus.cc:385] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.762009 29201 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.762693 29199 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.763567 29199 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.765699 29199 ts_tablet_manager.cc:1434] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.004s
I20260709 13:10:18.766722 29199 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:18.768473 29201 ts_tablet_manager.cc:1434] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:10:18.769768 29201 tablet_bootstrap.cc:492] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.772270 29199 tablet_bootstrap.cc:654] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.772950 29200 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.773533 29200 ts_tablet_manager.cc:1403] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.045s	user 0.004s	sys 0.016s
I20260709 13:10:18.777258 29200 raft_consensus.cc:359] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.778210 29201 tablet_bootstrap.cc:654] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.780203 29200 raft_consensus.cc:385] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.780613 29200 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.781884 29200 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.785686 29201 tablet_bootstrap.cc:492] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.786223 29201 ts_tablet_manager.cc:1403] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.001s
I20260709 13:10:18.787721 29200 ts_tablet_manager.cc:1434] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:10:18.788916 29200 tablet_bootstrap.cc:492] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:18.789480 29201 raft_consensus.cc:359] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.795014 29199 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.795585 29199 ts_tablet_manager.cc:1403] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.007s
I20260709 13:10:18.797163 29200 tablet_bootstrap.cc:654] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.798797 29199 raft_consensus.cc:359] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.799561 29199 raft_consensus.cc:385] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.799901 29199 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.801195 29199 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.801618 29207 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.802510 29200 tablet_bootstrap.cc:492] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.803145 29200 ts_tablet_manager.cc:1403] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:10:18.805898 29199 ts_tablet_manager.cc:1434] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.004s
I20260709 13:10:18.806254 29200 raft_consensus.cc:359] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.806892 29199 tablet_bootstrap.cc:492] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:18.806944 29200 raft_consensus.cc:385] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.807332 29200 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.802392 29201 raft_consensus.cc:385] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.802122 29207 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.808233 29201 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.808187 29200 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.809234 29201 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.809801 29207 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.811305 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:18.813728 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:18.814052 29201 ts_tablet_manager.cc:1434] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.004s
I20260709 13:10:18.814095 29199 tablet_bootstrap.cc:654] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.814368 29159 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:18.815140 29201 tablet_bootstrap.cc:492] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.819231 28975 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:18.819288 29200 ts_tablet_manager.cc:1434] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:10:18.811892 29085 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:18.821947 29201 tablet_bootstrap.cc:654] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.826067 29199 tablet_bootstrap.cc:492] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.826591 29199 ts_tablet_manager.cc:1403] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.020s	user 0.000s	sys 0.017s
I20260709 13:10:18.826889 29207 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.827414 29207 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.828527 29200 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:18.829821 29199 raft_consensus.cc:359] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.827829 29207 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.830633 29199 raft_consensus.cc:385] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.831035 29199 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.832142 29199 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.833528 29201 tablet_bootstrap.cc:492] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.834021 29201 ts_tablet_manager.cc:1403] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.001s
I20260709 13:10:18.834760 29199 ts_tablet_manager.cc:1434] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:18.835855 29199 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:18.836458 29200 tablet_bootstrap.cc:654] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.844266 29199 tablet_bootstrap.cc:654] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.845459 29207 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.848340 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:18.848940 29085 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.850685 29200 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.844331 29201 raft_consensus.cc:359] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.851513 29200 ts_tablet_manager.cc:1403] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.005s
I20260709 13:10:18.851687 29201 raft_consensus.cc:385] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.851940 29199 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.852106 29201 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.852478 29199 ts_tablet_manager.cc:1403] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.006s
I20260709 13:10:18.852909 29201 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.854657 29200 raft_consensus.cc:359] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.855376 29200 raft_consensus.cc:385] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.855458 29201 ts_tablet_manager.cc:1434] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:10:18.855322 29199 raft_consensus.cc:359] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.855902 29200 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.856204 29199 raft_consensus.cc:385] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.856438 29205 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.856587 29199 raft_consensus.cc:740] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.856769 29200 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.856925 29205 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.857429 29199 consensus_queue.cc:260] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.859035 29205 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:18.859201 29201 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.859860 29199 ts_tablet_manager.cc:1434] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:18.860873 29199 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:18.861039 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:18.861586 29200 ts_tablet_manager.cc:1434] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
W20260709 13:10:18.862475 29122 leader_election.cc:343] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.867288 29199 tablet_bootstrap.cc:654] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.868067 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:18.869824 29125 leader_election.cc:343] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:18.870235 29125 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:18.871302 29200 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:18.871779 29207 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.873019 29201 tablet_bootstrap.cc:654] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.875850 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:18.876646 29159 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.879434 29200 tablet_bootstrap.cc:654] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.882771 29199 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.883196 29199 ts_tablet_manager.cc:1403] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.011s
I20260709 13:10:18.886534 29205 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:18.886271 29199 raft_consensus.cc:359] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.887120 29199 raft_consensus.cc:385] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.887482 29199 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.888315 29199 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.890642 29199 ts_tablet_manager.cc:1434] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:18.891798 29199 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:18.892428 29201 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.892867 29201 ts_tablet_manager.cc:1403] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.006s
I20260709 13:10:18.895419 29201 raft_consensus.cc:359] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.896055 29201 raft_consensus.cc:385] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.896324 29201 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.896934 29201 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.898471 29199 tablet_bootstrap.cc:654] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.898860 29201 ts_tablet_manager.cc:1434] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:18.899683 29201 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.900528 29200 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.901089 29200 ts_tablet_manager.cc:1403] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.000s
I20260709 13:10:18.905064 29201 tablet_bootstrap.cc:654] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.907065 29200 raft_consensus.cc:359] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.907657 29200 raft_consensus.cc:385] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.907929 29200 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.908700 29085 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:18.904140 29159 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:18.911983 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:18.908787 29200 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.913336 29201 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.913261 29207 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:18.913853 29201 ts_tablet_manager.cc:1403] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:10:18.914129 29200 ts_tablet_manager.cc:1434] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:10:18.913872 29207 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:18.914950 29207 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.916550 29201 raft_consensus.cc:359] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.917066 29200 tablet_bootstrap.cc:492] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:18.917194 29201 raft_consensus.cc:385] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.917572 29201 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.918725 29201 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.921083 29199 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.921520 29199 ts_tablet_manager.cc:1403] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 13:10:18.924080 29199 raft_consensus.cc:359] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.924731 29199 raft_consensus.cc:385] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.925001 29199 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.925047 29200 tablet_bootstrap.cc:654] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.927181 29201 ts_tablet_manager.cc:1434] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:10:18.927718 29199 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.929684 29199 ts_tablet_manager.cc:1434] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:18.930507 29199 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:18.935998 29199 tablet_bootstrap.cc:654] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.939734 29201 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.942036 29200 tablet_bootstrap.cc:492] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:18.942564 29200 ts_tablet_manager.cc:1403] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.004s
I20260709 13:10:18.943964 29199 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.944478 29199 ts_tablet_manager.cc:1403] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:10:18.943696 28908 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:18.945433 29200 raft_consensus.cc:359] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.946144 29200 raft_consensus.cc:385] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.946465 29200 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.947255 29199 raft_consensus.cc:359] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.947396 29200 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.947986 29199 raft_consensus.cc:385] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.948323 29199 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.949369 29201 tablet_bootstrap.cc:654] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.949004 29199 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.949575 29200 ts_tablet_manager.cc:1434] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:18.950800 29200 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:18.951184 29199 ts_tablet_manager.cc:1434] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:18.951238 29207 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.951772 29207 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.952214 29199 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:18.954953 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:18.955545 29085 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:18.956758 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:18.957759 29241 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:18.958086 29241 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.958181 29200 tablet_bootstrap.cc:654] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.958460 29241 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.959239 29199 tablet_bootstrap.cc:654] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:18.964262 29207 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.966215 29241 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.967079 29206 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.967540 29206 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.969611 29206 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.970664 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:18.971561 29241 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:18.971521 29159 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:18.972275 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:18.973152 29010 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:18.970713 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:18.974365 29085 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.975852 29201 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:18.976317 29201 ts_tablet_manager.cc:1403] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.000s
I20260709 13:10:18.979281 29201 raft_consensus.cc:359] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.979969 29201 raft_consensus.cc:385] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:18.981072 29201 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:18.980768 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:18.981776 29201 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.983160 29050 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:18.985221 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:18.985652 29206 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.985797 29159 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:18.985976 29205 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.986481 29205 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:18.987598 29199 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:18.988118 29199 ts_tablet_manager.cc:1403] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.011s
I20260709 13:10:18.988556 29205 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:18.990578 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:18.983580 29201 ts_tablet_manager.cc:1434] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:18.997501 29201 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:18.999070 29206 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:19.000018 29010 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:19.001356 29199 raft_consensus.cc:359] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.002099 29199 raft_consensus.cc:385] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.002151 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:19.002449 29199 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.002573 29200 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:19.003047 29200 ts_tablet_manager.cc:1403] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.003s
I20260709 13:10:19.003183 29205 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.003558 29205 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.003890 29205 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.004132 29159 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:19.004632 29085 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:19.005697 29206 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.005409 28975 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:19.005739 29199 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.006321 29206 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.007668 29199 ts_tablet_manager.cc:1434] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:10:19.006633 29241 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.008425 29241 raft_consensus.cc:697] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:19.008556 29206 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.009372 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.009274 29241 consensus_queue.cc:237] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.010118 29010 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:19.011353 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:19.011894 29227 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.012611 29201 tablet_bootstrap.cc:654] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.012377 29227 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.016525 29227 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.016865 29207 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.017323 29207 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.018335 29200 raft_consensus.cc:359] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.018924 29200 raft_consensus.cc:385] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.019208 29200 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.019405 29207 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.019896 29200 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.020989 29257 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.021111 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.021432 29257 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.021816 29257 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:19.023057 28975 leader_election.cc:343] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.024555 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.025318 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.025959 29201 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:19.025959 29010 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:19.026641 29201 ts_tablet_manager.cc:1403] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.001s
I20260709 13:10:19.027588 29122 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:19.021440 29206 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.028595 29227 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.029013 29227 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.029431 29227 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.029644 29201 raft_consensus.cc:359] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.030011 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.030429 29201 raft_consensus.cc:385] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.030743 29201 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.030592 29199 tablet_bootstrap.cc:492] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:19.031199 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.032891 29084 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:19.034514 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:19.033133 29050 leader_election.cc:343] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.037627 29227 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.031478 29201 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.031723 29205 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:19.041957 28976 leader_election.cc:343] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.042495 28976 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:19.042779 29205 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.043964 29207 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:19.028580 29206 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.045121 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:19.045744 29010 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.046880 29206 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.048095 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.048332 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.049053 29159 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:19.049741 29262 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:19.050158 29048 leader_election.cc:343] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.050261 29262 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.053514 29201 ts_tablet_manager.cc:1434] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:10:19.053995 29207 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.054551 29201 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:19.054478 29207 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.056476 29205 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.061666 29227 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.062723 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:19.066861 29008 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.072731 29257 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.077446 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.079008 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.077728 29262 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.079523 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:19.082080 29085 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:19.082526 29006 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.085793 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.086459 29159 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:19.077718 29207 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.079632 29257 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.098038 29199 tablet_bootstrap.cc:654] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:19.083554 29122 leader_election.cc:343] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.097414 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:19.103699 29207 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.104377 29207 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.104741 29207 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:19.105252 29125 leader_election.cc:343] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.109102 29006 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:19.095238 29010 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:19.115057 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:19.115767 29257 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.116003 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:19.116447 29008 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:19.117434 29200 ts_tablet_manager.cc:1434] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.114s	user 0.017s	sys 0.027s
I20260709 13:10:19.116571 29257 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:19.120579 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:19.121418 29206 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.121903 29206 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.121392 29257 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.122241 29206 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.119242 29261 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.123647 29261 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:19.124954 29261 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.129985 29207 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.126473 28908 catalog_manager.cc:5697] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:19.131690 29206 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:19.127492 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 77033.2us
I20260709 13:10:19.135257 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:19.136861 29048 leader_election.cc:343] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.138255 29200 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:19.139607 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:19.145599 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:19.146489 29085 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.164225 29262 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.165009 29262 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.168666 29262 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.173867 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:19.195441 29264 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.196731 29260 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:19.197077 29050 leader_election.cc:343] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.197324 29260 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.199535 29206 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.199478 29264 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.201730 29200 tablet_bootstrap.cc:654] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:19.198518 29125 leader_election.cc:343] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.209630 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:19.210215 29158 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.211306 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.212303 29122 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:19.212823 29008 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:19.212108 29262 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.202584 29085 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:19.214337 29263 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.206971 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.216180 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.216578 29081 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:19.215503 29263 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:10:19.218098 28975 leader_election.cc:343] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.206235 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.202661 29264 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.219156 29201 tablet_bootstrap.cc:654] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.218170 29263 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.221392 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:19.213845 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.221340 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:19.224614 29083 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:19.224970 29262 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.225327 29262 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.225678 29262 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.225560 29084 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:19.223119 29264 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.223250 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:19.226831 29085 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.227298 29264 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.232401 29264 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.235328 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:19.235952 29159 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.235972 29125 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:19.237195 29205 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.237645 29205 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.238075 29205 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.238083 29207 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.254463 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.255335 29008 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:19.257274 29048 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:19.258399 29257 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.258905 29257 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.259335 29257 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.279706 29205 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.285887 28908 catalog_manager.cc:5697] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:19.290395 29260 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.305801 29199 tablet_bootstrap.cc:492] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:19.306469 29199 ts_tablet_manager.cc:1403] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.278s	user 0.016s	sys 0.005s
I20260709 13:10:19.302645 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.316553 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:19.324575 29157 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:19.325596 29257 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.326133 29200 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:19.326555 29200 ts_tablet_manager.cc:1403] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.189s	user 0.020s	sys 0.033s
I20260709 13:10:19.326864 29085 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:19.335742 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:19.327097 29199 raft_consensus.cc:359] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.336937 29199 raft_consensus.cc:385] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.337376 29199 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.336656 29207 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.338341 29207 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:19.327631 29257 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.328455 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:19.329737 29158 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
W20260709 13:10:19.331089 29122 leader_election.cc:343] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.329038 29262 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.339134 29008 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.341094 29264 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.341665 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:19.332863 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:19.342702 29157 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.337684 29200 raft_consensus.cc:359] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.344316 29200 raft_consensus.cc:385] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.344667 29200 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.345683 29200 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.313068 29205 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.338634 29199 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.339197 29207 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.343170 29264 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.362920 29200 ts_tablet_manager.cc:1434] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:10:19.348337 29201 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:19.432978 29201 ts_tablet_manager.cc:1403] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.379s	user 0.060s	sys 0.049s
I20260709 13:10:19.433084 29200 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:19.347786 29262 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.426154 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:19.442451 29081 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.442562 29008 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:19.432730 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:19.401292 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:19.365341 29260 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.337637 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:19.440331 29201 raft_consensus.cc:359] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.364583 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:19.444692 29201 raft_consensus.cc:385] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.446190 29201 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.404109 29241 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 13:10:19.443747 29159 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:19.450341 29201 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.446890 29009 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.447187 29085 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.447530 29084 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.454198 29201 ts_tablet_manager.cc:1434] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 13:10:19.448918 29048 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:19.448243 29260 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:19.462100 29264 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.467420 29199 ts_tablet_manager.cc:1434] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.161s	user 0.016s	sys 0.016s
I20260709 13:10:19.468449 29199 tablet_bootstrap.cc:492] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:19.462584 29081 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:19.471056 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:10:19.471276 28975 leader_election.cc:343] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.471879 29205 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.467146 29257 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.472373 29205 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:19.472499 29257 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:19.473613 29157 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:19.473218 29205 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.473245 29257 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.348973 28907 catalog_manager.cc:5697] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:19.484155 29085 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:19.484892 29264 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.489385 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:19.491650 29241 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.500360 29009 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:19.507344 29264 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.496817 29260 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.512732 29200 tablet_bootstrap.cc:654] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.513286 29199 tablet_bootstrap.cc:654] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.505797 29241 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:19.518620 29201 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:19.520941 29084 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:19.520273 29241 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.533519 29125 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:19.541980 29261 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.542547 29261 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:19.543450 29261 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.555248 29205 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.555928 29205 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.558853 29205 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.561267 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.568192 29009 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:19.570546 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.572584 29122 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:10:19.574448 29125 leader_election.cc:343] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.575495 29205 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.575948 29205 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.576345 29205 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.577814 28908 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:19.581117 29260 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.581683 29260 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.584625 29260 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.593333 29199 tablet_bootstrap.cc:492] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:19.594004 29199 ts_tablet_manager.cc:1403] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.126s	user 0.021s	sys 0.013s
I20260709 13:10:19.595518 29201 tablet_bootstrap.cc:654] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.597224 29199 raft_consensus.cc:359] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.597823 29200 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:19.597985 29199 raft_consensus.cc:385] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.598379 29199 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.598547 29200 ts_tablet_manager.cc:1403] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.166s	user 0.006s	sys 0.008s
I20260709 13:10:19.603601 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.605036 29009 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:19.618211 29199 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.619774 29205 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.623721 28907 catalog_manager.cc:5697] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:19.626740 28907 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:19.628925 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:19.629590 29009 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.638190 29199 ts_tablet_manager.cc:1434] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.001s
I20260709 13:10:19.639532 29199 tablet_bootstrap.cc:492] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:19.640571 29262 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.616657 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:19.653606 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:19.655371 29205 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.659406 29200 raft_consensus.cc:359] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.659128 29009 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:19.661792 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:19.662488 29122 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:19.663456 29262 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.666000 29201 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:19.661063 29200 raft_consensus.cc:385] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.669864 29201 ts_tablet_manager.cc:1403] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.152s	user 0.016s	sys 0.018s
W20260709 13:10:19.663477 29125 leader_election.cc:343] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.673139 29277 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.676880 29277 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.664014 29262 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:19.679109 29262 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.682057 29200 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.682855 29050 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:19.683771 29260 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.684898 29277 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.682925 29200 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.687621 29200 ts_tablet_manager.cc:1434] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.000s
I20260709 13:10:19.687599 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.688526 29157 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:19.688737 29200 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:19.693559 28908 catalog_manager.cc:5697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:19.695178 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.693557 29199 tablet_bootstrap.cc:654] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.697263 28906 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:19.703954 28906 catalog_manager.cc:5697] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:19.705422 29260 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:19.706058 28906 catalog_manager.cc:5697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:19.707101 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:19.710448 29201 raft_consensus.cc:359] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.710961 29201 raft_consensus.cc:385] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.711211 29201 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.717259 29260 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.717875 29205 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.718942 29261 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.725163 29295 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.732785 29200 tablet_bootstrap.cc:654] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.734666 29201 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:19.750164 29050 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:19.759992 29201 ts_tablet_manager.cc:1434] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.003s
I20260709 13:10:19.761063 29201 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:19.763972 29295 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:19.757292 29263 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.769407 29201 tablet_bootstrap.cc:654] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.772284 29199 tablet_bootstrap.cc:492] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:19.772751 29199 ts_tablet_manager.cc:1403] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.134s	user 0.014s	sys 0.001s
I20260709 13:10:19.778384 29199 raft_consensus.cc:359] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.779098 29199 raft_consensus.cc:385] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.779376 29199 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.780071 29199 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.782157 29199 ts_tablet_manager.cc:1434] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:10:19.781729 28907 catalog_manager.cc:5697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:19.784282 29200 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:19.784735 29200 ts_tablet_manager.cc:1403] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.096s	user 0.010s	sys 0.006s
I20260709 13:10:19.787251 29200 raft_consensus.cc:359] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.801115 29200 raft_consensus.cc:385] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.801401 29200 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.802019 29200 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.802920 29199 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:19.811512 29200 ts_tablet_manager.cc:1434] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.001s
I20260709 13:10:19.811666 29201 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:19.812105 29201 ts_tablet_manager.cc:1403] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.051s	user 0.024s	sys 0.016s
I20260709 13:10:19.812501 29200 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:19.814626 29201 raft_consensus.cc:359] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.815326 29201 raft_consensus.cc:385] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.815621 29201 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.816324 29201 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.818189 29199 tablet_bootstrap.cc:654] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.818329 29201 ts_tablet_manager.cc:1434] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:19.819545 29201 tablet_bootstrap.cc:492] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:19.822044 29200 tablet_bootstrap.cc:654] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.826599 29199 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:19.826964 29201 tablet_bootstrap.cc:654] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.827029 29199 ts_tablet_manager.cc:1403] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.002s
I20260709 13:10:19.830286 29199 raft_consensus.cc:359] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.831758 29199 raft_consensus.cc:385] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.832208 29199 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.833288 29199 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.836181 29199 ts_tablet_manager.cc:1434] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.004s
I20260709 13:10:19.838183 29199 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:19.839568 29201 tablet_bootstrap.cc:492] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:19.840258 29201 ts_tablet_manager.cc:1403] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:10:19.844234 29199 tablet_bootstrap.cc:654] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.843779 29201 raft_consensus.cc:359] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.844736 29201 raft_consensus.cc:385] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.845130 29201 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.846083 29295 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.846730 29295 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.847455 29201 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.850183 29295 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.850378 29201 ts_tablet_manager.cc:1434] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:19.851356 29201 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:19.851830 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.852218 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.852864 29157 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
W20260709 13:10:19.853260 28976 leader_election.cc:343] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.856199 29200 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:19.856647 29200 ts_tablet_manager.cc:1403] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.044s	user 0.023s	sys 0.007s
I20260709 13:10:19.858296 28975 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:19.859444 29200 raft_consensus.cc:359] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.860145 29200 raft_consensus.cc:385] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.860411 29200 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.861307 29200 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.862885 29201 tablet_bootstrap.cc:654] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.863435 29200 ts_tablet_manager.cc:1434] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:10:19.864722 29200 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:19.860435 29295 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.865891 29295 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.866302 29295 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.872974 29200 tablet_bootstrap.cc:654] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.873984 29295 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.876784 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:10:19.878137 28976 leader_election.cc:343] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.880673 29199 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:19.881176 29199 ts_tablet_manager.cc:1403] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.043s	user 0.006s	sys 0.004s
I20260709 13:10:19.884485 29199 raft_consensus.cc:359] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.885167 29199 raft_consensus.cc:385] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.885438 29199 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.886085 29199 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.893306 29201 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:19.893440 29199 ts_tablet_manager.cc:1434] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:10:19.894068 29201 ts_tablet_manager.cc:1403] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.019s
I20260709 13:10:19.894783 29199 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:19.897594 29201 raft_consensus.cc:359] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.898479 29201 raft_consensus.cc:385] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.898813 29201 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.899832 29201 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.901890 29199 tablet_bootstrap.cc:654] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.905476 29206 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.905957 29206 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.906265 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:19.906765 29157 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.908022 29206 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.912555 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:19.913968 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.914717 29008 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:19.915047 29263 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.915442 29206 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.915706 29263 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.915513 29277 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.919359 29157 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:19.898977 29295 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.919894 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:19.920862 29009 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:19.921569 29048 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:19.922423 29201 ts_tablet_manager.cc:1434] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 13:10:19.922694 29206 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.922788 29122 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:19.923337 29201 tablet_bootstrap.cc:492] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:19.926545 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.928264 29262 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:10:19.928445 29125 leader_election.cc:343] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:19.928705 29262 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.929335 29262 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.931380 29200 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:19.931777 29201 tablet_bootstrap.cc:654] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.931850 29200 ts_tablet_manager.cc:1403] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.067s	user 0.025s	sys 0.039s
I20260709 13:10:19.932060 29295 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:19.934779 29200 raft_consensus.cc:359] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.935477 29200 raft_consensus.cc:385] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.935786 29200 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.936553 29263 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.936561 29200 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.938858 29200 ts_tablet_manager.cc:1434] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:19.939730 29200 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:19.945869 29200 tablet_bootstrap.cc:654] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.945690 28975 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:19.946998 29262 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.950596 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:19.951156 29009 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.957263 29207 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.957983 29207 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.960135 29200 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:19.960848 29200 ts_tablet_manager.cc:1403] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:10:19.961206 29207 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:19.963275 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:19.963927 29081 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:19.963658 29200 raft_consensus.cc:359] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.964506 29200 raft_consensus.cc:385] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:19.964810 29200 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:19.965552 29200 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.967942 29200 ts_tablet_manager.cc:1434] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:19.968914 29200 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:19.947481 29295 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:19.972283 29295 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:19.972743 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:19.973158 29295 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:19.974120 29278 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:19.974617 29278 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.975065 29278 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:19.977530 29263 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:19.981891 29200 tablet_bootstrap.cc:654] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:19.985518 29206 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:19.990362 29260 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:20.005049 29262 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:19.995970 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:19.997268 28975 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:20.006000 29156 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:20.007906 29260 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.008617 29260 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.012754 29260 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.014963 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:20.016973 29050 leader_election.cc:343] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.021582 29009 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:20.023382 29122 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:20.026708 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.028335 29262 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.029383 29262 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:20.030326 29262 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:20.037067 29048 leader_election.cc:343] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.037688 29048 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:20.038844 29206 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:19.998162 29199 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:19.998787 29263 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:20.002712 29278 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.003757 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:20.049805 29277 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:10:20.059644 29085 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.072955 29200 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.073427 29200 ts_tablet_manager.cc:1403] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.105s	user 0.029s	sys 0.041s
I20260709 13:10:20.075982 29200 raft_consensus.cc:359] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.076675 29200 raft_consensus.cc:385] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.065624 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:20.077557 29156 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.077805 29085 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:20.042449 29199 ts_tablet_manager.cc:1403] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.148s	user 0.016s	sys 0.000s
I20260709 13:10:20.081570 29199 raft_consensus.cc:359] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.082111 29199 raft_consensus.cc:385] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.082429 29199 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.083267 29156 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:20.066203 29010 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.061383 29201 tablet_bootstrap.cc:492] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.084411 28975 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:20.085036 29201 ts_tablet_manager.cc:1403] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.162s	user 0.052s	sys 0.057s
I20260709 13:10:20.066591 29260 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:10:20.076956 29200 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.085728 29277 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.086298 29277 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:20.086975 29200 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.087219 29277 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.091434 29200 ts_tablet_manager.cc:1434] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 13:10:20.076926 28906 catalog_manager.cc:5697] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:20.067791 29278 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.089455 29201 raft_consensus.cc:359] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.093534 29201 raft_consensus.cc:385] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.083073 29199 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.093856 29201 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.094599 29201 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.095469 29199 ts_tablet_manager.cc:1434] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:10:20.095283 29206 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.096349 29199 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:20.099318 29201 ts_tablet_manager.cc:1434] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:10:20.100421 29201 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.102239 28907 catalog_manager.cc:5697] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:20.107043 29201 tablet_bootstrap.cc:654] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.106762 29263 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.114773 29200 tablet_bootstrap.cc:492] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:20.137352 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:20.138103 29081 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.138686 29262 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:20.152524 29081 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:20.166494 29200 tablet_bootstrap.cc:654] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.170948 29262 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.171864 29199 tablet_bootstrap.cc:654] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.183317 29200 tablet_bootstrap.cc:492] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.183162 29261 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.194108 29200 ts_tablet_manager.cc:1403] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.080s	user 0.010s	sys 0.017s
I20260709 13:10:20.188974 28906 catalog_manager.cc:5697] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:20.204293 29201 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.204864 29201 ts_tablet_manager.cc:1403] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.105s	user 0.010s	sys 0.007s
I20260709 13:10:20.207721 29201 raft_consensus.cc:359] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.208514 29201 raft_consensus.cc:385] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.208827 29201 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.209642 29201 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.211140 29200 raft_consensus.cc:359] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.212106 29200 raft_consensus.cc:385] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.212543 29201 ts_tablet_manager.cc:1434] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:20.213629 29201 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.216146 29263 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.216776 29263 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.220077 29263 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.221098 29201 tablet_bootstrap.cc:654] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.220973 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.221542 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.222375 29085 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
W20260709 13:10:20.222572 29122 leader_election.cc:343] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.223791 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:20.226116 29263 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.226598 29263 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.227046 29263 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.212463 29200 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.232331 29200 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.235251 29200 ts_tablet_manager.cc:1434] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:10:20.237183 29201 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.237744 29201 ts_tablet_manager.cc:1403] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.012s
I20260709 13:10:20.238672 29200 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:20.240578 29201 raft_consensus.cc:359] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.241304 29201 raft_consensus.cc:385] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.241641 29201 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.242462 29201 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.244298 29201 ts_tablet_manager.cc:1434] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:20.246546 29201 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.253505 29201 tablet_bootstrap.cc:654] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.255903 29263 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.262306 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:20.264114 29227 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.264832 29227 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.267802 29201 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.268528 29201 ts_tablet_manager.cc:1403] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.009s
I20260709 13:10:20.268565 29227 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.274736 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:20.275884 29122 leader_election.cc:343] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.277616 29201 raft_consensus.cc:359] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.278429 29201 raft_consensus.cc:385] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.278659 29201 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.279197 29201 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.281196 29201 ts_tablet_manager.cc:1434] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:10:20.281963 29205 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.282408 29201 tablet_bootstrap.cc:492] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.282648 29205 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.285506 29205 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.288556 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.288985 29201 tablet_bootstrap.cc:654] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.289479 29008 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:20.277601 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.293164 29122 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:20.294559 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.296380 29205 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.296909 29205 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.296676 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.297422 29205 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.297801 29083 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:20.299736 29261 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:20.322590 29199 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:20.323076 29199 ts_tablet_manager.cc:1403] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.227s	user 0.050s	sys 0.084s
I20260709 13:10:20.323927 29200 tablet_bootstrap.cc:654] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.326220 29199 raft_consensus.cc:359] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.326915 29199 raft_consensus.cc:385] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.327170 29199 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.327301 29205 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.327947 29199 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.329299 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:20.330020 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:20.330729 29006 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.334121 29199 ts_tablet_manager.cc:1434] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:10:20.335090 29199 tablet_bootstrap.cc:492] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:20.330060 29083 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.342540 29201 tablet_bootstrap.cc:492] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.343053 29201 ts_tablet_manager.cc:1403] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.061s	user 0.027s	sys 0.027s
I20260709 13:10:20.345615 29201 raft_consensus.cc:359] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.346275 29201 raft_consensus.cc:385] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.346910 29201 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.347710 29201 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.353041 29199 tablet_bootstrap.cc:654] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.353129 29156 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.354223 29201 ts_tablet_manager.cc:1434] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.010s
I20260709 13:10:20.355198 29201 tablet_bootstrap.cc:492] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.355504 29006 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:20.357512 29083 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:20.358731 29157 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.359124 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:20.360741 29261 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.361129 29201 tablet_bootstrap.cc:654] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.361441 29261 raft_consensus.cc:697] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:20.362455 29261 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.364217 29263 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.365134 29122 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:20.367173 29263 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.368355 29263 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:20.373013 29263 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.374356 29205 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.380792 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:20.381780 29083 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.384599 29260 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.385258 29260 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.386044 29199 tablet_bootstrap.cc:492] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:20.386507 29199 ts_tablet_manager.cc:1403] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.008s
I20260709 13:10:20.390039 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.392244 29260 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.392851 29199 raft_consensus.cc:359] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.393528 29199 raft_consensus.cc:385] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:10:20.394060 29048 leader_election.cc:343] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.393841 29199 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.395800 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:20.396597 29156 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:20.396528 29199 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.398216 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:20.400447 29257 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.400892 29257 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.401261 29257 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.409783 29199 ts_tablet_manager.cc:1434] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:10:20.410758 29257 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.411500 29205 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.412009 29205 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.414170 29205 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.415194 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.415583 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:20.415616 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:20.416455 29125 leader_election.cc:343] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:20.417078 29122 leader_election.cc:343] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:20.417078 29048 leader_election.cc:343] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.417620 29122 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.415915 29201 tablet_bootstrap.cc:492] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.418481 29261 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:20.418845 29201 ts_tablet_manager.cc:1403] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.004s
I20260709 13:10:20.419613 29257 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.419822 29200 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.420384 29200 ts_tablet_manager.cc:1403] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.182s	user 0.019s	sys 0.004s
I20260709 13:10:20.421366 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:20.421969 29157 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.421777 29201 raft_consensus.cc:359] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.422577 29201 raft_consensus.cc:385] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.422883 29201 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.423537 29201 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.423728 29200 raft_consensus.cc:359] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.425798 29083 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:20.425853 29201 ts_tablet_manager.cc:1434] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:20.426645 29200 raft_consensus.cc:385] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.426951 29200 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.427081 29201 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.427619 29200 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.430555 29157 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:20.432260 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:20.433420 29201 tablet_bootstrap.cc:654] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.435410 29199 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:20.435652 29257 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.437294 29257 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:20.438123 29257 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.442381 29200 ts_tablet_manager.cc:1434] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:10:20.443372 29200 tablet_bootstrap.cc:492] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:20.444554 29199 tablet_bootstrap.cc:654] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.445406 28906 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:20.447022 28906 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:20.456485 29200 tablet_bootstrap.cc:654] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.461311 29201 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.461885 29201 ts_tablet_manager.cc:1403] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.035s	user 0.023s	sys 0.000s
I20260709 13:10:20.464704 29201 raft_consensus.cc:359] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.464515 28908 catalog_manager.cc:5697] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:20.465332 29201 raft_consensus.cc:385] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.465618 29201 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.466276 29201 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.472522 29201 ts_tablet_manager.cc:1434] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:10:20.473423 29201 tablet_bootstrap.cc:492] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.476651 29295 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.479120 29201 tablet_bootstrap.cc:654] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.480880 29263 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.481338 29263 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.483487 29263 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.487901 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.488561 29085 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:20.489320 29295 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.486259 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.490234 29006 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:20.491504 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:20.492329 29205 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.492645 29205 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.493026 29205 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.496399 29199 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:20.497078 29199 ts_tablet_manager.cc:1403] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.062s	user 0.010s	sys 0.006s
I20260709 13:10:20.502669 29205 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.505970 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:20.506636 29006 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.515472 29006 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:20.524580 29205 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.526068 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:20.527195 29263 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.527690 29263 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:20.528548 29263 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.529178 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:20.529901 29083 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.535363 29262 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.537945 29262 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.542505 29199 raft_consensus.cc:359] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.543265 29199 raft_consensus.cc:385] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.543531 29199 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.544134 29199 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.546929 29200 tablet_bootstrap.cc:492] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.547050 29199 ts_tablet_manager.cc:1434] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:10:20.548149 29199 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:20.547418 29200 ts_tablet_manager.cc:1403] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.104s	user 0.022s	sys 0.032s
I20260709 13:10:20.551030 29200 raft_consensus.cc:359] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.551762 29200 raft_consensus.cc:385] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.555609 29083 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:20.555876 29201 tablet_bootstrap.cc:492] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.565385 29201 ts_tablet_manager.cc:1403] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.092s	user 0.003s	sys 0.008s
I20260709 13:10:20.556223 29199 tablet_bootstrap.cc:654] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.557752 29295 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.566148 29295 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.568312 29295 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.568461 29201 raft_consensus.cc:359] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.569700 29201 raft_consensus.cc:385] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.569974 29201 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.570027 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.570705 29201 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:20.572088 28976 leader_election.cc:343] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.573159 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:20.573767 29157 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:20.575371 29201 ts_tablet_manager.cc:1434] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
W20260709 13:10:20.575486 28976 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:20.576306 29201 tablet_bootstrap.cc:492] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.564316 29200 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.577386 29200 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.580071 28906 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:20.583230 29278 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.583705 29201 tablet_bootstrap.cc:654] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.584146 29200 ts_tablet_manager.cc:1434] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 13:10:20.585029 29200 tablet_bootstrap.cc:492] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:20.577643 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.588244 29278 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.588621 29278 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.588960 29278 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.590606 29200 tablet_bootstrap.cc:654] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.608515 29201 tablet_bootstrap.cc:492] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.609007 29201 ts_tablet_manager.cc:1403] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.011s
I20260709 13:10:20.611593 29201 raft_consensus.cc:359] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.612145 29201 raft_consensus.cc:385] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.612385 29201 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.613090 29201 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.616882 29201 ts_tablet_manager.cc:1434] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:10:20.617805 29201 tablet_bootstrap.cc:492] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.623418 29201 tablet_bootstrap.cc:654] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.626470 29200 tablet_bootstrap.cc:492] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.626935 29200 ts_tablet_manager.cc:1403] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.042s	user 0.003s	sys 0.010s
I20260709 13:10:20.630259 29200 raft_consensus.cc:359] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.630959 29200 raft_consensus.cc:385] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.631219 29200 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.631911 29200 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.637768 29278 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.640262 29278 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.640502 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:10:20.641727 28976 leader_election.cc:343] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.636612 29200 ts_tablet_manager.cc:1434] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:10:20.643572 29200 tablet_bootstrap.cc:492] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:20.647107 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:20.647998 29157 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.649298 29200 tablet_bootstrap.cc:654] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.656690 29199 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:20.657184 29199 ts_tablet_manager.cc:1403] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.109s	user 0.021s	sys 0.011s
I20260709 13:10:20.659739 29199 raft_consensus.cc:359] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.660447 29199 raft_consensus.cc:385] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.660722 29199 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.662125 29199 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.665031 29199 ts_tablet_manager.cc:1434] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:20.666721 29199 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:20.678854 29199 tablet_bootstrap.cc:654] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.683472 29263 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.684068 29263 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.686148 29263 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.686887 29200 tablet_bootstrap.cc:492] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.687354 29200 ts_tablet_manager.cc:1403] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.001s
I20260709 13:10:20.687916 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.689982 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:20.690508 29125 leader_election.cc:343] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.691098 29263 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:20.691408 29122 leader_election.cc:343] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.691643 29263 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.689952 29200 raft_consensus.cc:359] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.693204 29200 raft_consensus.cc:385] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.693470 29200 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.695254 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:20.697088 29125 leader_election.cc:343] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.692101 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.699635 29262 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:20.700394 29206 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.700765 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.701089 29206 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:20.702796 29122 leader_election.cc:343] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.703347 29206 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.703480 29263 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:10:20.705025 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:20.705941 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.706673 29006 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:20.708386 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:20.703373 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.709983 29263 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:20.710245 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:20.711246 29156 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:20.713254 29200 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.713868 29157 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:20.715510 29200 ts_tablet_manager.cc:1434] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.002s
I20260709 13:10:20.715586 29206 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.716760 29200 tablet_bootstrap.cc:492] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:20.716744 29206 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.717286 29206 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.723681 29200 tablet_bootstrap.cc:654] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.724367 29206 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.726671 29206 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.729002 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:20.729741 29006 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.730984 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:20.731628 29157 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.734633 29263 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.735252 29263 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.729715 29201 tablet_bootstrap.cc:492] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.736595 29201 ts_tablet_manager.cc:1403] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.119s	user 0.040s	sys 0.043s
I20260709 13:10:20.739434 29201 raft_consensus.cc:359] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.740199 29201 raft_consensus.cc:385] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.739825 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.739765 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.740496 29201 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
W20260709 13:10:20.741503 29125 leader_election.cc:343] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.742079 29263 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:10:20.742208 29122 leader_election.cc:343] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.742473 29006 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:20.742774 29122 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.743813 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:20.744504 29263 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:20.744902 29206 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.745627 29206 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:20.746541 29206 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.748327 29201 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.750972 29201 ts_tablet_manager.cc:1434] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:10:20.753559 29201 tablet_bootstrap.cc:492] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.762086 29201 tablet_bootstrap.cc:654] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.775230 29006 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.778075 29263 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.785480 29263 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.787036 29263 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.786710 28908 catalog_manager.cc:5697] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:20.790789 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.791652 29277 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.792145 29277 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:20.792925 29277 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.796494 29157 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
W20260709 13:10:20.796542 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 80636.9us
I20260709 13:10:20.806135 29201 tablet_bootstrap.cc:492] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.806849 29201 ts_tablet_manager.cc:1403] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.004s
I20260709 13:10:20.812009 29205 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.815296 29201 raft_consensus.cc:359] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.816869 29201 raft_consensus.cc:385] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.817235 29201 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.819146 29201 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.821341 29201 ts_tablet_manager.cc:1434] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:10:20.822377 29201 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.823968 29200 tablet_bootstrap.cc:492] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.824455 29200 ts_tablet_manager.cc:1403] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.108s	user 0.020s	sys 0.021s
I20260709 13:10:20.827101 29200 raft_consensus.cc:359] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.827849 29200 raft_consensus.cc:385] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.833396 29199 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:20.833877 29199 ts_tablet_manager.cc:1403] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.168s	user 0.036s	sys 0.053s
I20260709 13:10:20.831230 29200 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.835433 29200 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.836551 29201 tablet_bootstrap.cc:654] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.840504 29260 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:20.841298 29200 ts_tablet_manager.cc:1434] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:10:20.842577 29200 tablet_bootstrap.cc:492] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:20.848781 29260 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.849351 29260 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.850214 29200 tablet_bootstrap.cc:654] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.850939 28908 catalog_manager.cc:5697] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:20.852180 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.836499 29199 raft_consensus.cc:359] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.854329 29199 raft_consensus.cc:385] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.854648 29199 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.854743 29260 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.855453 29199 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.856564 29262 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.857062 29262 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.859127 29262 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:20.860538 29277 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.860471 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.861099 29277 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.861850 29199 ts_tablet_manager.cc:1434] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
W20260709 13:10:20.862494 29122 leader_election.cc:343] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.862986 29199 tablet_bootstrap.cc:492] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:20.863978 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:20.864868 29156 raft_consensus.cc:2393] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 1: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:10:20.865746 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:20.867086 28976 leader_election.cc:343] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.869040 29277 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.869262 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:20.870159 29156 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:20.871366 28975 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.872758 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.872802 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:20.874949 29257 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.875542 29277 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.877775 29277 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.882326 29200 tablet_bootstrap.cc:492] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.882822 29200 ts_tablet_manager.cc:1403] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.012s
W20260709 13:10:20.880578 29125 leader_election.cc:343] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.878211 29277 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.884438 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.889897 29199 tablet_bootstrap.cc:654] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.890038 29260 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.890748 29262 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:20.897091 29257 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:20.892583 29260 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.901093 29260 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.901906 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.900245 29277 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.905548 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:20.906695 29201 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.907167 29201 ts_tablet_manager.cc:1403] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.085s	user 0.026s	sys 0.014s
I20260709 13:10:20.909387 29200 raft_consensus.cc:359] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.910112 29200 raft_consensus.cc:385] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.910409 29200 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.911041 29200 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.913048 29200 ts_tablet_manager.cc:1434] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:10:20.912866 29050 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:20.914022 29200 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
W20260709 13:10:20.915097 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:20.909680 29201 raft_consensus.cc:359] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.916688 29201 raft_consensus.cc:385] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.916980 29201 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.914106 29257 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:20.918205 29201 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.920832 29201 ts_tablet_manager.cc:1434] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:10:20.921166 29200 tablet_bootstrap.cc:654] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.921830 29201 tablet_bootstrap.cc:492] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.922093 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:10:20.923554 28976 leader_election.cc:343] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.925583 29277 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.925849 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:20.926409 29157 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.931142 29201 tablet_bootstrap.cc:654] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.934782 29157 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:20.937837 28975 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:20.939154 29277 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:20.939659 29277 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:20.940622 29277 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.944948 29260 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.945637 29260 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.946636 29199 tablet_bootstrap.cc:492] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:20.947450 29199 ts_tablet_manager.cc:1403] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.008s
I20260709 13:10:20.948863 29205 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.949369 29205 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.949563 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.951450 29205 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:10:20.951545 29048 leader_election.cc:343] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.952515 29199 raft_consensus.cc:359] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.954402 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:20.954627 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:20.955372 29260 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.955747 29085 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:20.953609 29199 raft_consensus.cc:385] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.956746 29199 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.957264 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:20.957840 29157 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:20.957746 29199 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.959021 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:20.959856 29205 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.960242 29205 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.960574 29205 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.960739 29200 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:20.960726 29201 tablet_bootstrap.cc:492] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:20.961371 29200 ts_tablet_manager.cc:1403] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.009s
I20260709 13:10:20.961398 29201 ts_tablet_manager.cc:1403] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.012s
I20260709 13:10:20.964151 29201 raft_consensus.cc:359] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.964309 29200 raft_consensus.cc:359] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.964998 29201 raft_consensus.cc:385] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.965005 29200 raft_consensus.cc:385] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:20.965565 29200 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.966539 29200 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.966799 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:20.967676 29257 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:20.968077 29257 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.968415 29257 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:20.968858 29200 ts_tablet_manager.cc:1434] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:10:20.969647 29199 ts_tablet_manager.cc:1434] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:10:20.970561 29199 tablet_bootstrap.cc:492] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:20.965559 29201 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:20.975102 29257 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.975819 29199 tablet_bootstrap.cc:654] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.975459 29200 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:20.977689 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:20.975862 29201 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.981323 29262 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:20.981942 29122 leader_election.cc:343] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:20.982836 29048 leader_election.cc:343] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:20.985075 29201 ts_tablet_manager.cc:1434] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:10:20.985880 29257 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:20.985858 29200 tablet_bootstrap.cc:654] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.986785 29201 tablet_bootstrap.cc:492] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:20.989341 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:20.992681 29205 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.993901 29157 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:20.995329 29257 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.995834 29257 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:20.997153 29201 tablet_bootstrap.cc:654] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:20.997817 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:20.999711 29206 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.002287 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.002774 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:21.003826 29048 leader_election.cc:343] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.004889 29257 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:21.009191 29122 leader_election.cc:343] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.009666 29201 tablet_bootstrap.cc:492] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.010237 29201 ts_tablet_manager.cc:1403] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.003s
I20260709 13:10:21.009713 28908 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:21.012120 29200 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:21.012616 29200 ts_tablet_manager.cc:1403] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.002s
I20260709 13:10:21.013495 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.013551 29201 raft_consensus.cc:359] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.014276 29201 raft_consensus.cc:385] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.014705 29201 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.015443 29201 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.015595 29200 raft_consensus.cc:359] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.014254 29157 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:21.016829 29200 raft_consensus.cc:385] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.017172 29200 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.017776 29201 ts_tablet_manager.cc:1434] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:21.017843 29200 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.018818 29201 tablet_bootstrap.cc:492] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:21.020444 29200 ts_tablet_manager.cc:1434] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:21.021445 29200 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:21.025188 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:21.025624 29201 tablet_bootstrap.cc:654] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.026418 29205 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.027123 29200 tablet_bootstrap.cc:654] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.028247 29257 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.028733 29257 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.029170 29257 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.037643 29257 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.039863 29201 tablet_bootstrap.cc:492] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.040405 29201 ts_tablet_manager.cc:1403] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.008s
I20260709 13:10:21.040606 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:21.041944 29048 leader_election.cc:343] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.043004 29201 raft_consensus.cc:359] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.043670 29201 raft_consensus.cc:385] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.043967 29201 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.044893 29199 tablet_bootstrap.cc:492] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:21.043874 29205 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:21.051729 29200 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:21.052244 29200 ts_tablet_manager.cc:1403] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.022s
I20260709 13:10:21.045497 29199 ts_tablet_manager.cc:1403] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.004s
I20260709 13:10:21.053314 29257 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.049533 29085 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.062184 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:21.062785 29157 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.055424 29200 raft_consensus.cc:359] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.058561 29199 raft_consensus.cc:359] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.068758 29199 raft_consensus.cc:385] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.069182 29199 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.069504 29263 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.069983 29263 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.070259 29199 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.045270 29201 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.073068 29199 ts_tablet_manager.cc:1434] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 13:10:21.073340 29201 ts_tablet_manager.cc:1434] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.001s
I20260709 13:10:21.060631 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:21.074054 29199 tablet_bootstrap.cc:492] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:21.074270 29201 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:21.074795 29200 raft_consensus.cc:385] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.075212 29200 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.075959 29200 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.068133 29261 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.078835 29261 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.079731 29200 ts_tablet_manager.cc:1434] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:10:21.081336 29081 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:21.086076 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.074584 29263 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.089826 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.092113 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.093338 29257 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.093943 29257 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.094880 29263 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.097195 29200 tablet_bootstrap.cc:492] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:21.098238 29261 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:10:21.103854 29125 leader_election.cc:343] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.112429 29206 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.112428 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.117403 29257 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:21.118374 29125 leader_election.cc:343] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.121912 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.120106 29206 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.123138 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.123315 29199 tablet_bootstrap.cc:654] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.108686 29157 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:21.126561 29156 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
W20260709 13:10:21.126752 29048 leader_election.cc:343] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.130412 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:21.132392 29260 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.132761 29260 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.133055 29260 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.132901 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:21.138468 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:21.140005 29048 leader_election.cc:343] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:21.129503 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 43720us
I20260709 13:10:21.141395 29206 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.136286 29257 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:21.144647 29257 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:21.145520 29257 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:21.155437 29122 leader_election.cc:343] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.156193 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.158530 29010 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:21.159749 29200 tablet_bootstrap.cc:654] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.158353 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.162333 29261 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.162513 29201 tablet_bootstrap.cc:654] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.166199 29260 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.167642 29122 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.170260 29261 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.171185 29261 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.171552 29261 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.172693 29199 tablet_bootstrap.cc:492] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:21.173193 29199 ts_tablet_manager.cc:1403] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.099s	user 0.003s	sys 0.022s
I20260709 13:10:21.174224 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:21.175917 29048 leader_election.cc:343] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.176120 29199 raft_consensus.cc:359] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.176831 29199 raft_consensus.cc:385] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.176934 29260 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.177126 29199 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.177837 29199 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.179996 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.180960 29199 ts_tablet_manager.cc:1434] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:10:21.181735 29199 tablet_bootstrap.cc:492] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:21.181797 29261 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.184020 29261 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.186278 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:21.186870 29006 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.191828 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:21.192456 29156 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.198596 29006 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:21.199250 29156 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:21.200084 29122 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:21.200912 29263 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:21.201408 29263 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:21.202140 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:10:21.202369 29050 leader_election.cc:343] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.202304 29263 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.202920 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:21.203655 29260 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.204617 29199 tablet_bootstrap.cc:654] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.205566 29201 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.206005 29201 ts_tablet_manager.cc:1403] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.132s	user 0.005s	sys 0.006s
I20260709 13:10:21.205838 29260 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:21.207779 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:21.208626 29257 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:21.209518 29257 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:10:21.210546 29125 leader_election.cc:343] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.211544 29200 tablet_bootstrap.cc:492] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:21.212126 29200 ts_tablet_manager.cc:1403] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.132s	user 0.014s	sys 0.006s
I20260709 13:10:21.210770 29257 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.214337 29199 tablet_bootstrap.cc:492] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:21.214869 29199 ts_tablet_manager.cc:1403] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.002s
I20260709 13:10:21.209219 29201 raft_consensus.cc:359] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.218672 29201 raft_consensus.cc:385] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.219884 29199 raft_consensus.cc:359] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.220724 29199 raft_consensus.cc:385] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.221081 29199 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.221987 29199 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.222195 29206 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:21.223907 29200 raft_consensus.cc:359] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.224594 29200 raft_consensus.cc:385] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.224838 29200 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.225486 29200 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.218967 29201 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.227408 29201 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.229737 29201 ts_tablet_manager.cc:1434] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 13:10:21.230635 29201 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:21.238660 29200 ts_tablet_manager.cc:1434] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:10:21.239190 29199 ts_tablet_manager.cc:1434] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:10:21.240097 29199 tablet_bootstrap.cc:492] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:21.242475 29201 tablet_bootstrap.cc:654] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.239379 29200 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:21.246026 28906 catalog_manager.cc:5697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:21.250571 29200 tablet_bootstrap.cc:654] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.258037 28908 catalog_manager.cc:5697] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:21.261772 28908 catalog_manager.cc:5697] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:21.273835 29200 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:21.274536 29199 tablet_bootstrap.cc:654] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.274518 29200 ts_tablet_manager.cc:1403] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.007s
I20260709 13:10:21.277170 29201 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.277779 29201 ts_tablet_manager.cc:1403] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.014s
I20260709 13:10:21.279102 29200 raft_consensus.cc:359] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.279971 29200 raft_consensus.cc:385] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.280364 29200 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.281203 29200 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.281200 29201 raft_consensus.cc:359] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.282325 29201 raft_consensus.cc:385] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.282692 29201 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.284188 29200 ts_tablet_manager.cc:1434] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:21.283681 29201 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.285336 29200 tablet_bootstrap.cc:492] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:21.288132 29257 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.288774 29257 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.291445 29257 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.292766 29200 tablet_bootstrap.cc:654] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.293213 29263 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.293766 29263 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.294201 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:21.297008 29048 leader_election.cc:343] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.297387 29263 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.299516 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.300204 29085 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:21.301182 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.302008 29156 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:21.303576 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:21.305955 29122 leader_election.cc:343] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.307766 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:21.309176 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:21.310781 29261 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.311331 29261 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.311682 29261 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.312136 29277 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:21.316254 29206 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.316753 29206 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.317356 29385 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.317725 29385 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.318068 29385 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.318851 29206 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.321594 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.322397 29156 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 1: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:10:21.324837 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.326283 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:21.327399 29206 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.327018 29261 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.330623 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:21.334614 29122 leader_election.cc:343] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.335832 29385 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.338872 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:21.340297 29048 leader_election.cc:343] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.287618 29201 ts_tablet_manager.cc:1434] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:10:21.343017 29201 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:21.344765 29206 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.348348 29199 tablet_bootstrap.cc:492] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:21.348919 29199 ts_tablet_manager.cc:1403] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.109s	user 0.034s	sys 0.006s
I20260709 13:10:21.349277 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:21.349895 29083 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.351470 29199 raft_consensus.cc:359] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.352887 29385 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.354154 29278 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.357750 29199 raft_consensus.cc:385] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.358385 29199 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.357355 29278 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.359891 29277 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:21.360293 29278 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.362316 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.363180 29263 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.363641 29263 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.365295 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:21.365902 29156 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.368269 29261 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.368430 29083 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:21.369474 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:21.371308 29199 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.376252 29199 ts_tablet_manager.cc:1434] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:10:21.370265 29261 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:21.377240 29261 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:21.378110 29261 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.377182 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.381449 29199 tablet_bootstrap.cc:492] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:21.386746 29263 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.391716 29156 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:21.392839 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:21.395002 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.396579 29200 tablet_bootstrap.cc:492] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:21.397028 29200 ts_tablet_manager.cc:1403] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.112s	user 0.028s	sys 0.038s
I20260709 13:10:21.398682 29006 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.400352 29200 raft_consensus.cc:359] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.401046 29200 raft_consensus.cc:385] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.401307 29200 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.401980 29200 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.403537 29200 ts_tablet_manager.cc:1434] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
W20260709 13:10:21.389031 29122 leader_election.cc:343] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:21.404225 29125 leader_election.cc:343] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.404316 29385 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:21.404866 29385 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:21.405351 29201 tablet_bootstrap.cc:654] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.408365 29200 tablet_bootstrap.cc:492] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:21.408663 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.410494 28908 catalog_manager.cc:5697] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:21.414649 29261 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.426230 29199 tablet_bootstrap.cc:654] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.429198 29385 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.435144 29401 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.435679 29401 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.443535 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.450968 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.451467 29401 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.461089 29278 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.463124 29201 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.462975 29278 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.463907 29201 ts_tablet_manager.cc:1403] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.121s	user 0.025s	sys 0.010s
I20260709 13:10:21.465992 29278 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.467228 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.468360 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:21.472275 28976 leader_election.cc:343] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.472426 29200 tablet_bootstrap.cc:654] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.472678 29201 raft_consensus.cc:359] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.473421 29201 raft_consensus.cc:385] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.473703 29201 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.468313 29157 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:21.481355 29201 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.483839 29201 ts_tablet_manager.cc:1434] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:10:21.484822 29201 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:21.502322 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.503530 28907 catalog_manager.cc:5697] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:21.507418 29200 tablet_bootstrap.cc:492] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:21.507979 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.508829 29278 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.509028 29200 ts_tablet_manager.cc:1403] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.102s	user 0.018s	sys 0.008s
I20260709 13:10:21.513085 29201 tablet_bootstrap.cc:654] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.552431 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.553797 29401 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.553855 29385 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.554531 29385 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:21.574400 29406 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:10:21.516013 29278 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:21.587860 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.589010 29006 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:21.594373 29200 raft_consensus.cc:359] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.595031 29385 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.595144 29200 raft_consensus.cc:385] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.595561 29200 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.596529 29200 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.599022 29200 ts_tablet_manager.cc:1434] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.090s	user 0.006s	sys 0.001s
W20260709 13:10:21.599988 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 47726.9us
W20260709 13:10:21.601094 28975 leader_election.cc:343] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.601502 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.602641 29401 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.604592 29278 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.608724 29200 tablet_bootstrap.cc:492] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:21.612700 29199 tablet_bootstrap.cc:492] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:21.613207 29199 ts_tablet_manager.cc:1403] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.232s	user 0.030s	sys 0.035s
I20260709 13:10:21.615010 29206 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:21.619163 29385 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:21.620132 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.622274 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:21.623718 29206 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.630363 29401 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:21.629889 29199 raft_consensus.cc:359] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.634686 29199 raft_consensus.cc:385] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.635028 29199 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.635725 29199 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.637825 29199 ts_tablet_manager.cc:1434] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:10:21.638823 29199 tablet_bootstrap.cc:492] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:21.640584 29200 tablet_bootstrap.cc:654] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.637856 29206 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.645155 29199 tablet_bootstrap.cc:654] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.657897 29201 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.658411 29201 ts_tablet_manager.cc:1403] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.174s	user 0.009s	sys 0.011s
I20260709 13:10:21.680696 29201 raft_consensus.cc:359] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.681396 29201 raft_consensus.cc:385] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.681677 29201 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.686417 29201 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.680150 29261 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.689779 29261 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.691653 29261 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.695502 29201 ts_tablet_manager.cc:1434] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:10:21.696455 29201 tablet_bootstrap.cc:492] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:21.698100 29278 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.698607 29278 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.700958 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.701556 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:21.714844 29125 leader_election.cc:343] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.723381 29261 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.725981 29261 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:21.728255 29122 leader_election.cc:343] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.728914 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.730165 29262 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:21.732478 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.733301 29083 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:21.735086 29010 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.734982 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:21.744508 29277 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.744628 29261 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.744958 29277 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.745360 29277 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.747939 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.737326 29278 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.748886 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:21.749485 29122 leader_election.cc:343] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:21.750125 29411 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:10:21.752331 29125 leader_election.cc:343] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.754593 29277 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.755172 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:21.756182 29261 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:21.757660 29413 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:10:21.760070 29277 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.760075 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:21.762789 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.768136 29083 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.768381 29159 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:21.765919 29201 tablet_bootstrap.cc:654] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.770859 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:21.771389 29159 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.781271 29200 tablet_bootstrap.cc:492] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:21.781704 29200 ts_tablet_manager.cc:1403] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.173s	user 0.010s	sys 0.002s
I20260709 13:10:21.775517 29083 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:21.784178 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:21.787356 29408 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:21.787860 29408 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:21.788583 29159 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:21.788707 29408 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.791636 29199 tablet_bootstrap.cc:492] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:21.792127 29199 ts_tablet_manager.cc:1403] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.154s	user 0.010s	sys 0.040s
I20260709 13:10:21.780321 29385 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:10:21.798617 29199 raft_consensus.cc:359] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.800741 29199 raft_consensus.cc:385] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.801158 29199 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.802138 29206 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:21.804482 29199 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.816303 29199 ts_tablet_manager.cc:1434] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:10:21.817248 29199 tablet_bootstrap.cc:492] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:21.812381 29200 raft_consensus.cc:359] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.821375 29200 raft_consensus.cc:385] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.821682 29200 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.822279 29200 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.831575 29200 ts_tablet_manager.cc:1434] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:10:21.832621 29200 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:21.836292 29261 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:21.836550 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:21.838658 29261 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:21.858946 29199 tablet_bootstrap.cc:654] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:21.860080 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 59020.2us
W20260709 13:10:21.862401 29122 consensus_peers.cc:597] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:21.863343 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 72113.7us
I20260709 13:10:21.864673 29201 tablet_bootstrap.cc:492] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.865314 29201 ts_tablet_manager.cc:1403] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.169s	user 0.025s	sys 0.023s
I20260709 13:10:21.869624 29201 raft_consensus.cc:359] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.870318 29201 raft_consensus.cc:385] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.870585 29201 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.871191 29201 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.872942 29200 tablet_bootstrap.cc:654] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.873436 29201 ts_tablet_manager.cc:1434] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:21.874519 29201 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:21.882282 29201 tablet_bootstrap.cc:654] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.887468 29199 tablet_bootstrap.cc:492] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:21.887939 29199 ts_tablet_manager.cc:1403] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.001s
I20260709 13:10:21.893711 29206 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.894233 29206 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.897300 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.898046 29010 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:21.898985 29206 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.899154 29201 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.899636 29201 ts_tablet_manager.cc:1403] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.014s
I20260709 13:10:21.900632 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.901365 29158 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:21.902668 29201 raft_consensus.cc:359] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.903604 29201 raft_consensus.cc:385] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.900480 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:21.903877 29201 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.904590 29201 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.904925 29206 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.905465 29206 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.905926 29206 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.906661 29201 ts_tablet_manager.cc:1434] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:21.907554 29201 tablet_bootstrap.cc:492] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:21.909933 29200 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:21.910432 29200 ts_tablet_manager.cc:1403] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.078s	user 0.006s	sys 0.003s
I20260709 13:10:21.912945 29200 raft_consensus.cc:359] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.913560 29200 raft_consensus.cc:385] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.913820 29200 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.914008 29201 tablet_bootstrap.cc:654] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:21.907691 29199 raft_consensus.cc:359] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.914458 29200 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.922808 29263 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.923497 29263 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.924155 29199 raft_consensus.cc:385] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:21.924602 29199 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:21.925396 29199 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.928328 29199 ts_tablet_manager.cc:1434] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 13:10:21.928483 29263 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.929672 29199 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:21.931218 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:21.932235 29083 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:21.932598 29200 ts_tablet_manager.cc:1434] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 13:10:21.934006 29200 tablet_bootstrap.cc:492] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:21.933856 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:21.935827 29261 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.936393 29261 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.936889 29261 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.948534 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:21.953119 29122 leader_election.cc:343] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.960253 29257 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:21.964526 29261 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.965617 29257 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.966136 29257 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.968544 29257 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:21.968958 29201 tablet_bootstrap.cc:492] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:21.969913 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:21.969738 29206 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.970366 29261 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:21.971354 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:21.971873 29262 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:21.977924 29048 consensus_peers.cc:597] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:21.963346 28906 catalog_manager.cc:5697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:21.975733 29201 ts_tablet_manager.cc:1403] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.068s	user 0.024s	sys 0.026s
I20260709 13:10:21.976665 29200 tablet_bootstrap.cc:654] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:21.982999 29122 leader_election.cc:343] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:21.983713 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:21.984452 29159 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:21.986426 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:21.987393 29257 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:21.987733 29257 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.988076 29257 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:21.986387 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:21.994163 29415 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.995030 29415 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:21.997311 29083 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.004590 29206 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.013929 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:22.014635 29157 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.007772 29415 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.019440 29257 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.023185 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.023878 29159 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:22.020032 29157 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.025242 29201 raft_consensus.cc:359] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.025846 29201 raft_consensus.cc:385] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.026126 29201 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.026823 29201 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.032403 29199 tablet_bootstrap.cc:654] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.035071 29200 tablet_bootstrap.cc:492] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:22.036453 29200 ts_tablet_manager.cc:1403] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.103s	user 0.020s	sys 0.010s
I20260709 13:10:22.037354 29257 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.035852 29201 ts_tablet_manager.cc:1434] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.000s
I20260709 13:10:22.038635 29201 tablet_bootstrap.cc:492] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
W20260709 13:10:22.005481 29048 leader_election.cc:343] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.077600 29262 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:22.078187 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 72939.9us
I20260709 13:10:22.097932 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:22.099388 29010 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.100821 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:22.101861 29158 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.079210 29262 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.112815 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.126436 29050 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.126770 29009 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:22.128014 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.129051 29415 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:22.129417 29206 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.129702 29415 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.130198 29206 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:22.130389 29415 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.131776 29206 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.137871 29262 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:22.123365 29199 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:22.143558 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:22.147037 29010 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
W20260709 13:10:22.148666 29048 leader_election.cc:343] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.147040 29158 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.153280 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:22.164000 29200 raft_consensus.cc:359] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.164836 29200 raft_consensus.cc:385] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.166328 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:22.168566 29122 leader_election.cc:343] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.174034 29260 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.174652 29260 raft_consensus.cc:697] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:22.165144 29200 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.194064 29201 tablet_bootstrap.cc:654] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.175913 29260 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.196681 29199 ts_tablet_manager.cc:1403] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.267s	user 0.012s	sys 0.004s
I20260709 13:10:22.186108 29200 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.205729 29083 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
W20260709 13:10:22.217638 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 64553.8us
I20260709 13:10:22.217371 29415 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.212343 29199 raft_consensus.cc:359] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.228143 29199 raft_consensus.cc:385] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.228482 29199 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.232095 29199 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.233935 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.250360 29419 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:22.243618 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:22.251492 29009 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.253793 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:22.254565 29158 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.257349 29419 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.257881 29419 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.242364 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:22.238200 29200 ts_tablet_manager.cc:1434] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.201s	user 0.001s	sys 0.008s
I20260709 13:10:22.262612 29263 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.244648 29415 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.264766 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.266337 29082 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:22.266975 29009 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.268292 29419 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.269071 29261 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.269651 29261 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.270025 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.267903 29263 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:22.271281 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.272449 29260 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.272235 29261 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:22.272688 29263 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.273963 29157 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:22.273016 29260 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:22.274109 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.279218 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:22.285326 29408 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.287303 29199 ts_tablet_manager.cc:1434] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.090s	user 0.010s	sys 0.000s
I20260709 13:10:22.286361 29260 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.293412 29201 tablet_bootstrap.cc:492] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:22.294004 29201 ts_tablet_manager.cc:1403] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.256s	user 0.016s	sys 0.001s
I20260709 13:10:22.294909 29158 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.297663 29200 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:22.299063 29199 tablet_bootstrap.cc:492] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
W20260709 13:10:22.287127 29125 leader_election.cc:343] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.302515 29125 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:22.304198 29262 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:22.308151 29122 leader_election.cc:343] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.308611 29201 raft_consensus.cc:359] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.309317 29201 raft_consensus.cc:385] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.309581 29201 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.310281 29201 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.314438 29201 ts_tablet_manager.cc:1434] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:10:22.315587 29408 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:22.316743 29201 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:22.323679 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.324642 29199 tablet_bootstrap.cc:654] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.328308 29200 tablet_bootstrap.cc:654] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.331969 29201 tablet_bootstrap.cc:654] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.333760 28908 catalog_manager.cc:5697] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:22.336881 29401 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:22.343027 29125 leader_election.cc:343] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.343487 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:22.345189 29262 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:22.360569 29201 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:22.361099 29201 ts_tablet_manager.cc:1403] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.000s
I20260709 13:10:22.360428 28908 catalog_manager.cc:5697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:22.378149 29201 raft_consensus.cc:359] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.378947 29201 raft_consensus.cc:385] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.379269 29201 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.380134 29201 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.388598 29201 ts_tablet_manager.cc:1434] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:10:22.389763 29201 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:22.402168 29200 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:22.405038 29419 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.403576 29199 tablet_bootstrap.cc:492] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:22.405781 29419 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.406533 29200 ts_tablet_manager.cc:1403] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.109s	user 0.016s	sys 0.004s
I20260709 13:10:22.407416 29201 tablet_bootstrap.cc:654] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.406262 29199 ts_tablet_manager.cc:1403] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.107s	user 0.010s	sys 0.018s
I20260709 13:10:22.412372 29200 raft_consensus.cc:359] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.413054 29200 raft_consensus.cc:385] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.413311 29200 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.409605 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.414037 29200 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.414547 29082 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:22.407958 29419 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.415784 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.417461 29200 ts_tablet_manager.cc:1434] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:10:22.417308 28975 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:22.418560 29200 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:22.418876 29419 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.421696 29199 raft_consensus.cc:359] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.427205 29199 raft_consensus.cc:385] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.427598 29199 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.428287 29199 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.428366 28906 catalog_manager.cc:5697] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:22.429807 29199 ts_tablet_manager.cc:1434] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 13:10:22.430799 29199 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:22.436975 29199 tablet_bootstrap.cc:654] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.439992 29201 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:22.440469 29201 ts_tablet_manager.cc:1403] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.051s	user 0.006s	sys 0.007s
I20260709 13:10:22.426328 29419 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:22.442641 28908 catalog_manager.cc:5697] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:22.443223 29201 raft_consensus.cc:359] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.443853 29201 raft_consensus.cc:385] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.449131 29201 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.449899 29201 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.426652 29200 tablet_bootstrap.cc:654] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.454847 29201 ts_tablet_manager.cc:1434] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:10:22.455910 29201 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:22.456125 29199 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:22.456574 29199 ts_tablet_manager.cc:1403] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.004s
I20260709 13:10:22.459635 29199 raft_consensus.cc:359] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.460381 29199 raft_consensus.cc:385] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.460651 29199 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.461411 29199 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.461979 29201 tablet_bootstrap.cc:654] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.468670 29199 ts_tablet_manager.cc:1434] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:10:22.470170 29199 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:22.479367 29200 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:22.479926 29200 ts_tablet_manager.cc:1403] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.008s
I20260709 13:10:22.482555 29200 raft_consensus.cc:359] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.483299 29200 raft_consensus.cc:385] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.483628 29200 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.485087 29199 tablet_bootstrap.cc:654] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.484354 29200 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.492172 29200 ts_tablet_manager.cc:1434] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:10:22.493516 29200 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:22.494216 29201 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:22.494689 29201 ts_tablet_manager.cc:1403] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.013s
I20260709 13:10:22.497188 29201 raft_consensus.cc:359] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.497844 29201 raft_consensus.cc:385] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.500953 29201 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.501696 29201 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.502424 29200 tablet_bootstrap.cc:654] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.502705 29199 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:22.503193 29199 ts_tablet_manager.cc:1403] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.016s
I20260709 13:10:22.506265 29199 raft_consensus.cc:359] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.507249 29199 raft_consensus.cc:385] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.507615 29201 ts_tablet_manager.cc:1434] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:10:22.507645 29199 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.508963 29199 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.513778 29200 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:22.514262 29200 ts_tablet_manager.cc:1403] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 13:10:22.516830 29200 raft_consensus.cc:359] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.517472 29200 raft_consensus.cc:385] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.517748 29200 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.518500 29200 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.522580 29199 ts_tablet_manager.cc:1434] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:10:22.523105 29201 tablet_bootstrap.cc:492] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:22.523372 29200 ts_tablet_manager.cc:1434] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:10:22.524294 29200 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:22.529004 29201 tablet_bootstrap.cc:654] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.529755 29200 tablet_bootstrap.cc:654] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.530705 29199 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:22.541388 29199 tablet_bootstrap.cc:654] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.542790 29200 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:22.543291 29200 ts_tablet_manager.cc:1403] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 13:10:22.546561 29260 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.547214 29260 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.551373 29260 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.552871 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.553843 29158 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:22.554185 29200 raft_consensus.cc:359] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.555130 29200 raft_consensus.cc:385] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.555457 29200 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.556687 29200 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.558714 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.559851 29260 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:22.560186 29200 ts_tablet_manager.cc:1434] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.007s
I20260709 13:10:22.560400 29260 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.560961 29260 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.562541 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.561632 29200 tablet_bootstrap.cc:492] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:22.563465 29006 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:22.564534 29419 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.565054 29419 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.567924 29419 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.570120 29199 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:22.570647 29199 ts_tablet_manager.cc:1403] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.016s
I20260709 13:10:22.570801 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.571347 29200 tablet_bootstrap.cc:654] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.573307 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.573503 29199 raft_consensus.cc:359] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.574338 29199 raft_consensus.cc:385] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.574229 29158 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:22.574858 29199 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.571697 29082 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:22.577463 29199 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.578601 29263 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.575879 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:22.579177 29263 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.579895 29439 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.580319 29439 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.581284 29263 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:22.582393 29439 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.583029 29419 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:22.583277 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.583446 29419 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.583941 29419 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.585357 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:22.585644 29122 leader_election.cc:343] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.585958 29157 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:22.587302 29200 tablet_bootstrap.cc:492] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:22.587764 29200 ts_tablet_manager.cc:1403] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.000s
I20260709 13:10:22.587528 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.594868 29419 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.595661 29082 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:22.597836 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:22.598362 29157 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.598526 29199 ts_tablet_manager.cc:1434] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.002s
I20260709 13:10:22.599859 29439 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:22.600215 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:22.596804 29419 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.599617 29199 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:22.590818 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.594731 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.602468 29439 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:22.602809 29439 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.603102 29439 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.607074 29260 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.609814 29439 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:22.611424 29048 leader_election.cc:343] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.611742 29439 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.613327 29125 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:22.590456 29200 raft_consensus.cc:359] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.615407 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:22.616763 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:22.623245 29263 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:22.623657 29263 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.623708 29159 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.624099 29263 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.631091 29263 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.600778 29082 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.621858 29157 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
W20260709 13:10:22.632555 29048 leader_election.cc:343] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.633723 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:22.634701 29419 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.637779 29419 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:22.638924 29419 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.641327 29200 raft_consensus.cc:385] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.642253 29257 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.649859 29257 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.652580 29257 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.653168 29200 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.646018 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:22.654731 29081 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.656008 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.656785 29157 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.642879 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:22.653848 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.657891 29009 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
W20260709 13:10:22.658052 29122 leader_election.cc:343] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.654960 29200 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.659104 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.659914 29257 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:22.660100 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:22.660738 29009 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.661725 29200 ts_tablet_manager.cc:1434] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.000s
I20260709 13:10:22.660291 29257 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.662075 29081 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:22.652215 29439 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:22.664891 29257 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:22.645622 29263 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:22.685926 29125 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:22.687072 29263 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.687544 29263 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:22.688153 29401 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.688270 29263 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.688660 29401 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.689685 29257 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.669714 29260 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.672169 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:22.622293 29199 tablet_bootstrap.cc:654] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.691743 29158 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.673364 29200 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:22.693051 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.693854 29156 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:22.637707 29159 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.678224 29009 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.662179 29082 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:22.681169 28906 catalog_manager.cc:5697] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:22.690845 29401 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.699906 29158 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.701229 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.704210 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:22.682389 29201 tablet_bootstrap.cc:492] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:22.698334 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:22.696630 29260 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.704737 29006 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:22.705370 29201 ts_tablet_manager.cc:1403] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.197s	user 0.041s	sys 0.073s
I20260709 13:10:22.705647 29439 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.706614 29439 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:22.707329 29439 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.708606 29201 raft_consensus.cc:359] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.709245 29201 raft_consensus.cc:385] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.709486 29201 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.710116 29201 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.711931 29006 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:22.715317 28976 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:22.718411 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.720567 29440 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.721099 29440 raft_consensus.cc:697] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:22.714250 29260 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.721938 29440 consensus_queue.cc:237] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.725001 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:22.730360 29158 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:22.716382 29257 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.736903 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.724169 29401 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.739326 29446 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.742542 29158 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:22.746455 29441 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:22.721591 29201 ts_tablet_manager.cc:1434] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:10:22.747481 29446 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.750442 29201 tablet_bootstrap.cc:492] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:22.752034 29446 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:22.753110 29441 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:22.753517 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.752719 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:22.760798 29122 leader_election.cc:343] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.763257 29260 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.764395 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:22.764765 29125 leader_election.cc:343] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.754081 29441 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.766659 29200 tablet_bootstrap.cc:654] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.758263 28907 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:22.765802 29125 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:22.771453 29446 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:22.773371 29446 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:22.773993 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.778031 29199 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:22.778527 29199 ts_tablet_manager.cc:1403] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.179s	user 0.016s	sys 0.000s
I20260709 13:10:22.766510 29158 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:22.784389 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:10:22.785259 29048 leader_election.cc:343] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.786484 29260 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:22.788033 29260 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.791356 29260 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.791631 29199 raft_consensus.cc:359] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.793246 29199 raft_consensus.cc:385] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.793474 29199 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.794590 29199 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.798905 29261 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.800508 29261 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.803390 29401 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:22.824474 29199 ts_tablet_manager.cc:1434] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:10:22.826841 29199 tablet_bootstrap.cc:492] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
W20260709 13:10:22.855917 29464 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 13:10:22.856453 29010 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:22.864995 29469 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:10:22.866778 29261 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:22.867038 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.868223 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:22.877242 29125 leader_election.cc:343] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.877167 29257 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:22.857522 29419 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.858673 29440 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:22.882870 29122 leader_election.cc:343] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:22.883920 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:22.886896 29262 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:22.882171 29440 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.881286 29419 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.898198 29415 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:22.898940 29201 tablet_bootstrap.cc:654] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.899040 29419 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.888854 29446 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:10:22.896350 29440 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.905965 29200 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:22.906476 29200 ts_tablet_manager.cc:1403] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.243s	user 0.018s	sys 0.043s
I20260709 13:10:22.908524 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:22.918064 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:22.926996 28906 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:22.911341 29260 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.929454 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.930132 29159 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:22.930426 29260 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:22.931830 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:22.937898 29050 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:22.949496 29440 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:22.950080 29440 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.950547 29440 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.964627 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:22.965508 29157 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.955328 29439 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:10:22.956920 28975 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:22.985595 29200 raft_consensus.cc:359] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.986809 29200 raft_consensus.cc:385] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.988713 29200 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.991233 29200 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:22.995222 29473 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:22.999497 29199 tablet_bootstrap.cc:654] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.000969 29200 ts_tablet_manager.cc:1434] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.001s
I20260709 13:10:23.002753 29157 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:22.991664 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:23.005498 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:23.007347 29260 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.007843 29260 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:10:23.001858 29048 leader_election.cc:343] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.015851 29200 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:23.021101 29441 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:23.024857 29048 leader_election.cc:343] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.008674 29260 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.051198 29201 tablet_bootstrap.cc:492] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:23.051735 29201 ts_tablet_manager.cc:1403] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.302s	user 0.017s	sys 0.030s
I20260709 13:10:23.055111 29009 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.055922 29440 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.081954 29201 raft_consensus.cc:359] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.082736 29201 raft_consensus.cc:385] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.083005 29201 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.083600 29201 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.088316 29201 ts_tablet_manager.cc:1434] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:10:23.089759 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:23.091074 29048 leader_election.cc:343] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.099215 29201 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:23.091361 29440 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.122092 29200 tablet_bootstrap.cc:654] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:23.124269 29122 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:23.126734 29476 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:23.127611 29201 tablet_bootstrap.cc:654] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.131377 29262 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.133112 29199 tablet_bootstrap.cc:492] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:23.131986 29262 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.133731 29199 ts_tablet_manager.cc:1403] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.307s	user 0.017s	sys 0.009s
I20260709 13:10:23.140030 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:23.140941 29158 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.143970 28907 catalog_manager.cc:5697] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:23.145617 28907 catalog_manager.cc:5697] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 2, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 2 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:23.149864 29158 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:23.149977 29262 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:23.154125 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:23.155347 29199 raft_consensus.cc:359] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.156354 29199 raft_consensus.cc:385] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:10:23.156687 29125 leader_election.cc:343] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.156711 29199 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.157390 29263 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.157796 29263 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.155831 29262 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:23.161630 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:23.166584 29122 leader_election.cc:343] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.167202 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:23.171918 29263 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:23.172077 29199 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.169836 29262 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.175019 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:23.169385 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:23.169245 29048 consensus_peers.cc:597] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:23.176993 29125 leader_election.cc:343] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:23.167719 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:10:23.178025 29122 leader_election.cc:343] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.179894 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:23.180787 29050 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:23.178146 29257 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:10:23.182706 29263 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.183727 29440 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.188707 29440 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:23.189548 29440 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.184478 29157 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:23.194216 29199 ts_tablet_manager.cc:1434] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.001s
I20260709 13:10:23.197471 29257 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:23.200649 29199 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:23.193459 29260 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:10:23.232738 29200 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:23.233263 29200 ts_tablet_manager.cc:1403] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.224s	user 0.014s	sys 0.007s
I20260709 13:10:23.253054 29385 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.253573 29385 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.256291 28906 catalog_manager.cc:5697] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:23.243774 29200 raft_consensus.cc:359] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.258631 29200 raft_consensus.cc:385] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.258955 29200 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.255793 29385 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.262089 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.261399 29200 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.262322 29473 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:23.264989 29007 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:23.272615 29201 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:23.273116 29201 ts_tablet_manager.cc:1403] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.174s	user 0.015s	sys 0.007s
I20260709 13:10:23.274703 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:23.277806 29260 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.278326 29260 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.267385 29415 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:23.282665 29260 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:23.284411 29048 leader_election.cc:343] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.286866 29200 ts_tablet_manager.cc:1434] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.007s
I20260709 13:10:23.288935 29200 tablet_bootstrap.cc:492] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:23.292027 29199 tablet_bootstrap.cc:654] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:23.293426 29050 leader_election.cc:343] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.294019 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:23.294167 29201 raft_consensus.cc:359] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.294910 29201 raft_consensus.cc:385] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.295193 29201 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.295871 29201 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.291512 29440 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:10:23.297633 29415 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.303436 29201 ts_tablet_manager.cc:1434] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.002s
I20260709 13:10:23.305184 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:23.303320 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.305487 29201 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
W20260709 13:10:23.308166 29048 leader_election.cc:343] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:23.311530 29050 leader_election.cc:343] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.312036 29050 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:23.313335 29415 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.356817 29199 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:23.357306 29199 ts_tablet_manager.cc:1403] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.157s	user 0.013s	sys 0.004s
I20260709 13:10:23.358501 29200 tablet_bootstrap.cc:654] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.365091 29201 tablet_bootstrap.cc:654] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.365739 29199 raft_consensus.cc:359] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.366691 29199 raft_consensus.cc:385] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.367072 29199 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.368080 29199 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.377053 28906 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:23.378539 29199 ts_tablet_manager.cc:1434] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:10:23.379577 29199 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:23.403426 29199 tablet_bootstrap.cc:654] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.405627 29200 tablet_bootstrap.cc:492] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:23.408294 29200 ts_tablet_manager.cc:1403] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.120s	user 0.016s	sys 0.009s
I20260709 13:10:23.408396 29201 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:23.408875 29201 ts_tablet_manager.cc:1403] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.104s	user 0.012s	sys 0.007s
I20260709 13:10:23.411042 29200 raft_consensus.cc:359] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.411743 29200 raft_consensus.cc:385] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.412031 29200 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.411871 29201 raft_consensus.cc:359] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.412612 29201 raft_consensus.cc:385] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.412834 29200 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.412860 29201 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.414629 29201 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.415267 29200 ts_tablet_manager.cc:1434] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:23.416290 29200 tablet_bootstrap.cc:492] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:23.417425 29201 ts_tablet_manager.cc:1434] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:23.421875 29200 tablet_bootstrap.cc:654] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.422463 29201 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:23.429177 29201 tablet_bootstrap.cc:654] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.436069 29199 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:23.436738 29199 ts_tablet_manager.cc:1403] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.004s
I20260709 13:10:23.437318 29200 tablet_bootstrap.cc:492] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:23.437740 29200 ts_tablet_manager.cc:1403] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.003s
I20260709 13:10:23.440325 29200 raft_consensus.cc:359] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.441016 29200 raft_consensus.cc:385] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.441290 29200 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
W20260709 13:10:23.442297 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:23.442307 29200 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.442574 29199 raft_consensus.cc:359] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.443393 29201 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:23.443563 29199 raft_consensus.cc:385] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.443902 29201 ts_tablet_manager.cc:1403] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.011s
I20260709 13:10:23.443970 29199 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.444967 29199 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.446946 29201 raft_consensus.cc:359] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.447623 29201 raft_consensus.cc:385] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.447880 29201 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.448364 29200 ts_tablet_manager.cc:1434] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:23.448757 29201 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.449265 29200 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:23.451273 29201 ts_tablet_manager.cc:1434] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:23.452314 29201 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:23.455716 29200 tablet_bootstrap.cc:654] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.457279 29199 ts_tablet_manager.cc:1434] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 13:10:23.458266 29199 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:23.466400 29199 tablet_bootstrap.cc:654] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.468689 29201 tablet_bootstrap.cc:654] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.474462 29200 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:23.474968 29200 ts_tablet_manager.cc:1403] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.006s
I20260709 13:10:23.477986 29200 raft_consensus.cc:359] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.478616 29200 raft_consensus.cc:385] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.478868 29200 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.479472 29200 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.481902 29200 ts_tablet_manager.cc:1434] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:23.482733 29200 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:23.488185 29200 tablet_bootstrap.cc:654] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.493693 29201 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:23.494447 29201 ts_tablet_manager.cc:1403] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.008s
I20260709 13:10:23.494931 29440 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:23.498032 29201 raft_consensus.cc:359] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.498675 29201 raft_consensus.cc:385] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.498847 29201 raft_consensus.cc:740] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.499722 29201 consensus_queue.cc:260] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.502418 29201 ts_tablet_manager.cc:1434] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:10:23.507288 29199 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:23.528644 29200 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:23.534335 29199 ts_tablet_manager.cc:1403] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.008s
I20260709 13:10:23.534363 29200 ts_tablet_manager.cc:1403] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.004s
I20260709 13:10:23.535784 29201 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:23.535954 29446 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.524602 29262 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.536482 29446 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.537145 29262 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.537391 29200 raft_consensus.cc:359] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.538077 29200 raft_consensus.cc:385] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.538332 29200 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.538574 29446 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:23.539002 29200 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.534825 29419 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.541292 29440 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.541913 29419 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.542239 29440 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.544806 29440 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.544961 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:23.545397 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.540228 29262 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:23.547899 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.548612 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.549105 29201 tablet_bootstrap.cc:654] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.549614 29200 ts_tablet_manager.cc:1434] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
W20260709 13:10:23.551522 29122 leader_election.cc:343] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.551652 29419 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:23.551759 29048 leader_election.cc:343] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.554172 29200 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:23.554394 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:23.554595 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:23.556293 28975 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:23.557597 29125 leader_election.cc:343] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.558233 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:23.558166 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:23.558240 29476 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.562242 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:23.562278 29122 leader_election.cc:343] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.564006 29154 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:23.561290 29262 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.566366 29083 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:23.569619 29199 raft_consensus.cc:359] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.569765 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:23.572121 29261 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:23.572625 29261 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.572976 29008 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.572981 29261 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.578080 29200 tablet_bootstrap.cc:654] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.578083 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:23.583346 29385 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:23.583734 29385 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.584113 29385 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.600010 29476 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.600719 29476 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.604293 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:23.570417 29199 raft_consensus.cc:385] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.606655 29199 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.608426 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:23.606887 29476 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.609314 29154 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:23.609663 29199 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:23.610574 28976 leader_election.cc:343] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.614848 29199 ts_tablet_manager.cc:1434] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.080s	user 0.004s	sys 0.003s
I20260709 13:10:23.614904 28975 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:23.616132 29476 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:23.616992 29476 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.617323 29476 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.599855 29385 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.608139 29261 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.626925 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:23.627431 29261 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:23.628686 29385 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.630101 29199 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
W20260709 13:10:23.630215 29048 leader_election.cc:343] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.632473 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:23.633265 29005 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.636224 29201 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:23.636706 29201 ts_tablet_manager.cc:1403] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.101s	user 0.029s	sys 0.008s
I20260709 13:10:23.636826 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:23.637631 29081 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.638985 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:23.639613 29154 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.640340 29201 raft_consensus.cc:359] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.642127 29201 raft_consensus.cc:385] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.642428 29201 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.643131 29201 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.655205 29257 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.655776 29257 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.656546 29201 ts_tablet_manager.cc:1434] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:10:23.682842 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.684728 29257 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:23.685179 29048 leader_election.cc:343] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.689363 29081 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:23.664444 29260 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.691934 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:23.700408 29201 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:23.695367 29260 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.703298 29262 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.693857 29476 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.709468 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.710384 29200 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:23.703918 29262 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:23.711975 29262 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.711051 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:23.713955 29159 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:23.736603 29511 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.737391 29511 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.742623 29005 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:23.748492 29260 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.782559 29154 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:23.791283 29199 tablet_bootstrap.cc:654] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.794047 29476 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.796546 29200 ts_tablet_manager.cc:1403] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.243s	user 0.026s	sys 0.033s
I20260709 13:10:23.800189 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:23.796464 29511 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.800868 29082 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:23.803133 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:23.803814 29156 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.804492 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:23.808737 29201 tablet_bootstrap.cc:654] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:23.810053 29048 leader_election.cc:343] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.811751 29156 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:23.812151 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:23.814627 29401 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:23.815057 29401 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.815459 29401 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:23.816646 28976 leader_election.cc:343] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.817692 29261 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.818221 29261 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.819484 29050 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:23.820469 29440 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:23.820839 29440 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.821187 29440 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.821235 29261 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:23.826363 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:23.827069 29156 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:23.827652 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:23.828282 29154 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:23.829063 29401 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.830624 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.831318 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:23.834919 29257 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:23.835021 29385 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.836779 29385 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:10:23.841626 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:23.842710 29200 raft_consensus.cc:359] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.843453 29200 raft_consensus.cc:385] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.843765 29200 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.844477 29200 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.839542 29401 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.839334 29385 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.830574 29440 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.846283 28975 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:23.850167 29122 leader_election.cc:343] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.849483 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:23.852056 29083 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.849632 29511 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.853559 29511 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:23.854794 29440 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.854516 29511 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.859965 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:23.861995 29083 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:23.843111 29050 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:23.868767 29200 ts_tablet_manager.cc:1434] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.000s
I20260709 13:10:23.871176 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:23.871740 29157 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.872768 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:23.873368 29159 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.873863 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:23.879176 29199 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:23.878576 28908 catalog_manager.cc:5697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:23.879632 29199 ts_tablet_manager.cc:1403] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.250s	user 0.007s	sys 0.016s
I20260709 13:10:23.870226 29440 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:23.880165 29201 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:23.880395 29440 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.880674 29201 ts_tablet_manager.cc:1403] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.190s	user 0.012s	sys 0.007s
I20260709 13:10:23.880769 29440 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:23.871968 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:23.881953 29157 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
W20260709 13:10:23.882539 29125 leader_election.cc:343] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.883275 29125 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:23.883963 29262 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:23.886524 29476 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.889043 29200 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:23.888031 29201 raft_consensus.cc:359] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.890448 29201 raft_consensus.cc:385] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.890954 29201 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.886752 29159 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
W20260709 13:10:23.891513 29048 leader_election.cc:343] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.892482 29201 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.887753 29476 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:23.897096 29199 raft_consensus.cc:359] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.897886 29199 raft_consensus.cc:385] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:23.898201 29199 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:23.897745 29476 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.898886 29199 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.910097 29050 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:23.911055 29199 ts_tablet_manager.cc:1434] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.030s	user 0.000s	sys 0.008s
I20260709 13:10:23.912822 29201 ts_tablet_manager.cc:1434] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.001s
I20260709 13:10:23.910356 29260 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.914208 29257 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:23.915524 29260 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.919307 29201 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:23.923285 29199 tablet_bootstrap.cc:492] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:23.937347 29419 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.937908 29260 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:23.937952 29419 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.940598 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:23.942826 29048 leader_election.cc:343] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:23.966495 29262 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.979193 29262 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.984855 29262 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:23.986533 29257 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:23.978902 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:23.941151 29440 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.990240 29201 tablet_bootstrap.cc:654] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:23.987438 29257 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:23.997059 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:23.991789 29159 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:23.992486 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:24.005152 28906 catalog_manager.cc:5697] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.006119 29440 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:24.006870 29419 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:24.008121 29122 leader_election.cc:343] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.010814 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:24.011940 29125 leader_election.cc:343] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.012385 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:24.012670 29050 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:24.013067 29261 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.014865 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:24.016086 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:24.016719 29156 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:24.019064 29048 leader_election.cc:343] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.010409 28905 catalog_manager.cc:5697] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.020195 29260 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.024273 29260 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.025902 29260 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.027796 29200 tablet_bootstrap.cc:654] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.032691 29260 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.036388 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:24.037837 29048 leader_election.cc:343] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.038892 29260 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:24.041886 29199 tablet_bootstrap.cc:654] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.041637 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:24.042454 29159 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.047214 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:24.049386 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:24.055298 29419 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.065732 29156 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:24.075044 29050 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:24.076122 29257 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.076642 29257 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:24.077421 29257 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.071583 28906 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.097266 29201 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:24.097754 29201 ts_tablet_manager.cc:1403] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.184s	user 0.023s	sys 0.020s
I20260709 13:10:24.101272 29159 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:24.104907 29050 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:24.107591 29201 raft_consensus.cc:359] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.108343 29201 raft_consensus.cc:385] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.108664 29201 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.109295 29201 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.109491 29524 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.110683 29524 raft_consensus.cc:697] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:24.111398 29201 ts_tablet_manager.cc:1434] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:10:24.117081 29201 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:24.119906 29200 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:24.120411 29200 ts_tablet_manager.cc:1403] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.249s	user 0.020s	sys 0.015s
I20260709 13:10:24.111500 29524 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.123842 29199 tablet_bootstrap.cc:492] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:24.124575 29199 ts_tablet_manager.cc:1403] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.202s	user 0.014s	sys 0.007s
I20260709 13:10:24.130497 29201 tablet_bootstrap.cc:654] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.133371 29200 raft_consensus.cc:359] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.134087 29200 raft_consensus.cc:385] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.134357 29200 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.134972 29200 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.137321 29200 ts_tablet_manager.cc:1434] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:10:24.139158 29200 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:24.142941 29201 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:24.143524 29201 ts_tablet_manager.cc:1403] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.001s
I20260709 13:10:24.144906 29200 tablet_bootstrap.cc:654] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.146370 29201 raft_consensus.cc:359] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.147078 29201 raft_consensus.cc:385] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.147338 29201 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.148255 29201 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.150419 29201 ts_tablet_manager.cc:1434] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:10:24.151748 29201 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:24.152523 29200 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:24.152952 29200 ts_tablet_manager.cc:1403] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 13:10:24.153769 29199 raft_consensus.cc:359] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.154539 29199 raft_consensus.cc:385] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.154807 29199 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.155598 29199 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.155982 29200 raft_consensus.cc:359] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.156757 29200 raft_consensus.cc:385] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.157056 29200 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.157673 29200 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.159777 29200 ts_tablet_manager.cc:1434] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.003s
I20260709 13:10:24.160319 29201 tablet_bootstrap.cc:654] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.160737 29200 tablet_bootstrap.cc:492] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:24.171878 29199 ts_tablet_manager.cc:1434] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.000s
I20260709 13:10:24.172969 29199 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:24.172480 28906 catalog_manager.cc:5697] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.174409 28906 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.176759 28906 catalog_manager.cc:5697] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.191738 29199 tablet_bootstrap.cc:654] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.202625 29200 tablet_bootstrap.cc:654] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.205473 29263 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.205960 29263 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.207887 29263 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:24.209551 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:24.210836 29122 leader_election.cc:343] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.212291 29200 tablet_bootstrap.cc:492] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:24.212198 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:24.212893 29200 ts_tablet_manager.cc:1403] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.003s
I20260709 13:10:24.212894 29081 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:24.215378 29201 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:24.215292 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:24.215924 29201 ts_tablet_manager.cc:1403] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.029s
I20260709 13:10:24.216766 29199 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:24.217231 29199 ts_tablet_manager.cc:1403] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.001s
I20260709 13:10:24.217476 29261 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.217839 29261 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.218146 29261 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.218904 29201 raft_consensus.cc:359] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.219679 29201 raft_consensus.cc:385] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.220002 29201 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.219861 29199 raft_consensus.cc:359] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.220643 29199 raft_consensus.cc:385] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.220970 29199 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.221802 29199 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.222364 29200 raft_consensus.cc:359] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.223358 29200 raft_consensus.cc:385] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.223656 29200 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.224280 29199 ts_tablet_manager.cc:1434] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:10:24.224287 29200 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.220992 29201 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.231478 29199 tablet_bootstrap.cc:492] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:24.232173 29200 ts_tablet_manager.cc:1434] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:10:24.234706 29200 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:24.236550 29524 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.237026 29524 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.239042 29524 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:24.240568 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:24.246222 29201 ts_tablet_manager.cc:1434] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:10:24.247373 29201 tablet_bootstrap.cc:492] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
W20260709 13:10:24.241952 29048 leader_election.cc:343] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.250757 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:24.251394 29156 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:24.253293 29006 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.257754 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:24.259680 29199 tablet_bootstrap.cc:654] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.262076 29261 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.264240 29260 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:10:24.261745 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:24.264626 29260 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.264966 29260 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.266841 29201 tablet_bootstrap.cc:654] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.275239 29199 tablet_bootstrap.cc:492] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:24.275840 29199 ts_tablet_manager.cc:1403] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.011s
I20260709 13:10:24.277182 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:24.278543 29122 leader_election.cc:343] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.279806 29199 raft_consensus.cc:359] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.280885 29199 raft_consensus.cc:385] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.281381 29199 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.282506 29199 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.284466 29261 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:24.286440 29199 ts_tablet_manager.cc:1434] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:10:24.287922 29199 tablet_bootstrap.cc:492] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:24.292181 29261 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.292651 29261 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.295491 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:24.295697 29199 tablet_bootstrap.cc:654] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.296660 29201 tablet_bootstrap.cc:492] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:24.296828 29081 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.297207 29201 ts_tablet_manager.cc:1403] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.000s
I20260709 13:10:24.298653 29260 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.301234 29201 raft_consensus.cc:359] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.302197 29201 raft_consensus.cc:385] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.302564 29201 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.302601 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:24.304127 29048 leader_election.cc:343] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.303660 29201 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.305897 29081 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:24.307273 29201 ts_tablet_manager.cc:1434] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:10:24.308920 29201 tablet_bootstrap.cc:492] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:24.309396 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:24.310420 29263 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.310871 29263 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:24.311640 29263 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.323697 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:24.312314 29260 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:24.324559 29159 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.335480 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:24.336437 29200 tablet_bootstrap.cc:654] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:24.339881 29122 leader_election.cc:343] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.341693 29261 leader_election.cc:290] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:24.347190 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:24.347378 29159 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:24.347787 29005 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:24.348483 29125 leader_election.cc:343] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.348886 29125 leader_election.cc:304] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:24.351014 29201 tablet_bootstrap.cc:654] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.355993 29446 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.359907 29201 tablet_bootstrap.cc:492] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:24.354436 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:24.360929 29201 ts_tablet_manager.cc:1403] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.000s
I20260709 13:10:24.361433 29524 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.365789 29201 raft_consensus.cc:359] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.366817 29201 raft_consensus.cc:385] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.367194 29201 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.368695 29262 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:24.368183 29201 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.370596 29201 ts_tablet_manager.cc:1434] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:10:24.372115 29201 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:24.385629 29524 raft_consensus.cc:697] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:24.393594 29201 tablet_bootstrap.cc:654] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.387653 29524 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.395623 29476 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:24.399127 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44767.8us
I20260709 13:10:24.423810 29257 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.424391 29257 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.427371 29201 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:24.427927 29201 ts_tablet_manager.cc:1403] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.000s
I20260709 13:10:24.429636 29257 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:24.423720 29476 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.425446 29082 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.489904 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:24.438138 29476 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.504148 29201 raft_consensus.cc:359] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.504439 29446 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:24.504931 29201 raft_consensus.cc:385] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.505281 29201 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.503299 29440 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:10:24.505945 29201 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.519661 29201 ts_tablet_manager.cc:1434] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.000s
I20260709 13:10:24.523736 29010 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.541627 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:24.542222 29511 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:24.543483 29048 leader_election.cc:343] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:24.618121 29539 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:10:24.618690 29048 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:24.653467 29385 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.654081 29385 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:24.688169 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:24.689873 29050 leader_election.cc:343] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.690444 29199 tablet_bootstrap.cc:492] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:24.691828 29199 ts_tablet_manager.cc:1403] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.404s	user 0.066s	sys 0.126s
I20260709 13:10:24.690335 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:24.696817 29200 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:24.697372 29200 ts_tablet_manager.cc:1403] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.463s	user 0.017s	sys 0.001s
I20260709 13:10:24.698194 29261 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.698630 29201 tablet_bootstrap.cc:492] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:24.698796 29261 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:24.688764 29185 tablet.cc:2406] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260709 13:10:24.701207 29538 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.171s	user 0.016s	sys 0.009s
I20260709 13:10:24.691326 29008 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.725318 29415 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.735292 29263 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.736574 29263 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.786594 29263 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:10:24.793025 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 99623.5us
I20260709 13:10:24.794991 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:24.801036 29476 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:24.819932 29545 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:10:24.837823 29199 raft_consensus.cc:359] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.838908 29199 raft_consensus.cc:385] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.839267 29199 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.846079 29385 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:24.850780 29199 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:24.856002 29548 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:10:24.858750 29263 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.860136 29199 ts_tablet_manager.cc:1434] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.168s	user 0.007s	sys 0.002s
I20260709 13:10:24.859241 29263 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.827549 29556 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.053s
I20260709 13:10:24.864318 29200 raft_consensus.cc:359] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:24.866590 29200 raft_consensus.cc:385] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.866971 29200 raft_consensus.cc:740] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.867872 29200 consensus_queue.cc:260] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:24.874539 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 74610.8us
I20260709 13:10:24.877739 29263 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:10:24.861505 29547 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:10:24.882273 29199 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:24.892208 29201 tablet_bootstrap.cc:654] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.913223 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:24.914388 29155 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:24.946792 29558 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:10:24.998431 28906 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:24.999861 29544 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:25.000507 29544 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.012933 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:25.045387 29048 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:25.062337 29200 ts_tablet_manager.cc:1434] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.365s	user 0.024s	sys 0.025s
I20260709 13:10:25.064822 29261 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:25.048578 29545 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.065490 29261 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.068315 29200 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:25.067879 29261 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.070977 29545 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.073153 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:25.131875 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.287478 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.225366 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.322072 29261 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:25.331528 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:25.332510 29545 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:25.350378 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:25.353848 29557 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.354393 29557 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.374588 29201 tablet_bootstrap.cc:492] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:25.376504 29201 ts_tablet_manager.cc:1403] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.845s	user 0.031s	sys 0.027s
I20260709 13:10:25.377907 29199 tablet_bootstrap.cc:654] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.371723 29154 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:25.386960 29544 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:25.387966 29580 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.392549 29200 tablet_bootstrap.cc:654] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.406028 29557 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:25.408552 29048 consensus_peers.cc:597] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:25.408847 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:25.403479 28907 catalog_manager.cc:5697] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.419914 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.428558 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:25.445072 29048 leader_election.cc:343] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.447918 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:25.477689 29261 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.478363 29261 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.481318 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:25.481810 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:25.493726 29257 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.503456 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:25.529815 29201 raft_consensus.cc:359] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:25.497521 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 50755.3us
I20260709 13:10:25.533099 29201 raft_consensus.cc:385] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.533448 29201 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.530215 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:25.514734 29564 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:25.538452 29201 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.502802 29584 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:25.542409 29584 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:25.545491 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61879.5us
I20260709 13:10:25.546809 29524 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.559701 29200 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:25.560268 29200 ts_tablet_manager.cc:1403] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.492s	user 0.020s	sys 0.054s
I20260709 13:10:25.563202 29200 raft_consensus.cc:359] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.496717 29261 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:25.563974 29200 raft_consensus.cc:385] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.569558 29200 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.569806 29201 ts_tablet_manager.cc:1434] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.193s	user 0.014s	sys 0.001s
I20260709 13:10:25.570222 29200 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.571198 29201 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:25.577072 29200 ts_tablet_manager.cc:1434] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:10:25.578274 29201 tablet_bootstrap.cc:654] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.579027 29200 tablet_bootstrap.cc:492] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:25.590956 29201 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:25.591457 29201 ts_tablet_manager.cc:1403] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.009s
W20260709 13:10:25.592249 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 45609.3us
I20260709 13:10:25.592815 29200 tablet_bootstrap.cc:654] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.594070 29201 raft_consensus.cc:359] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.594789 29201 raft_consensus.cc:385] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.595075 29201 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.595983 29201 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.599275 29201 ts_tablet_manager.cc:1434] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
W20260709 13:10:25.600514 29122 leader_election.cc:343] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.601223 29201 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Bootstrap starting.
I20260709 13:10:25.607511 29440 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.611495 29200 tablet_bootstrap.cc:492] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:25.612138 29200 ts_tablet_manager.cc:1403] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.003s
W20260709 13:10:25.622255 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:10:25.617846 29122 leader_election.cc:343] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:25.638017 29122 leader_election.cc:343] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:25.638622 29122 leader_election.cc:343] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.636252 29201 tablet_bootstrap.cc:654] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.646031 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.648312 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.649571 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:25.650270 29125 leader_election.cc:343] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.650698 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:25.652303 29552 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:25.651940 29125 leader_election.cc:343] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.654001 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:25.612553 29584 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:25.642432 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:25.675027 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:25.675982 29153 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:25.663627 29419 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.663954 29552 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.703418 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:25.704324 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:25.705432 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.708621 29587 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:25.712148 29200 raft_consensus.cc:359] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.712878 29200 raft_consensus.cc:385] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.713204 29200 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.713652 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:25.709838 29125 leader_election.cc:343] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.715708 29200 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.716076 29125 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:25.716814 29263 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:25.718331 29125 leader_election.cc:343] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.718730 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:25.723217 29587 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:25.723548 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.709720 29552 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
W20260709 13:10:25.726244 29122 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:25.727435 29122 leader_election.cc:343] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.713204 29419 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.732743 29050 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:25.734354 29200 ts_tablet_manager.cc:1434] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.000s
I20260709 13:10:25.734843 29257 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.735206 29257 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.736191 29257 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.735468 29553 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:10:25.732666 29552 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:25.742440 29125 leader_election.cc:343] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.743158 29125 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:25.753108 29199 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:25.753592 29199 ts_tablet_manager.cc:1403] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.879s	user 0.012s	sys 0.045s
I20260709 13:10:25.754647 29200 tablet_bootstrap.cc:492] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:25.756721 29257 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.766824 29257 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:25.745169 29552 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.771341 29199 raft_consensus.cc:359] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.772197 29199 raft_consensus.cc:385] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.772482 29199 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.752452 29553 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.779557 29553 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.813769 29199 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.815554 29199 ts_tablet_manager.cc:1434] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.001s
I20260709 13:10:25.816779 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:25.830300 29199 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
W20260709 13:10:25.836529 29595 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:10:25.837060 29048 leader_election.cc:343] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.818847 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:25.844444 29263 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.845628 29263 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.847157 29158 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.857403 29263 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:25.844856 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:25.866199 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:25.872013 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:25.872871 29083 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:25.874089 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:25.875422 29151 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:25.884073 29263 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:10:25.884868 29048 leader_election.cc:343] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.886713 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:25.888032 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:25.888533 29200 tablet_bootstrap.cc:654] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:25.893263 29571 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.915853 29571 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.916256 29571 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:25.923101 29122 leader_election.cc:343] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:25.925921 29553 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:25.966076 29263 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.966586 29263 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.991137 29263 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:25.995878 29263 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:25.997346 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:25.926599 29557 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:26.003120 29600 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:10:26.031158 29597 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:10:26.008944 29544 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.042469 29544 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.052342 29199 tablet_bootstrap.cc:654] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.058254 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:26.078557 29544 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.080108 29158 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:26.080248 29552 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.080809 29552 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.085628 29552 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:26.088527 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:26.090687 29125 leader_election.cc:343] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:26.093394 29122 leader_election.cc:343] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.097847 29571 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.100880 29083 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:26.103209 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:26.104458 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:26.104748 29048 leader_election.cc:343] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.108881 29551 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.118841 29200 tablet_bootstrap.cc:492] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:26.119475 29200 ts_tablet_manager.cc:1403] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.372s	user 0.024s	sys 0.013s
I20260709 13:10:26.134756 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:26.007957 29557 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.152418 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:26.154492 29557 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.156795 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:26.164785 29048 leader_election.cc:343] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.177129 29200 raft_consensus.cc:359] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.178273 29200 raft_consensus.cc:385] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.178649 29200 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.190724 29050 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:10:26.197594 29122 leader_election.cc:343] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.200397 29588 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.185770 29571 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.198256 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:26.205638 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.212134 29263 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.204031 29588 raft_consensus.cc:697] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:26.225457 29201 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: No bootstrap required, opened a new log
I20260709 13:10:26.228098 29201 ts_tablet_manager.cc:1403] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Time spent bootstrapping tablet: real 0.627s	user 0.058s	sys 0.051s
I20260709 13:10:26.228322 29588 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:26.235318 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61088.4us
I20260709 13:10:26.240141 29009 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.240902 29201 raft_consensus.cc:359] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.241696 29201 raft_consensus.cc:385] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.241986 29201 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.242734 29201 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.244858 29201 ts_tablet_manager.cc:1434] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:10:26.256695 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.257310 29199 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:26.257779 29199 ts_tablet_manager.cc:1403] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.438s	user 0.025s	sys 0.020s
I20260709 13:10:26.257362 29150 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:26.295452 29199 raft_consensus.cc:359] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.296443 29199 raft_consensus.cc:385] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.296778 29199 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.297593 29199 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.317188 28975 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:26.323011 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:26.323945 29151 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:26.324963 29199 ts_tablet_manager.cc:1434] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.001s
I20260709 13:10:26.326221 29199 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:26.333527 29200 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.335107 29200 ts_tablet_manager.cc:1434] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.215s	user 0.008s	sys 0.004s
I20260709 13:10:26.335979 29200 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:26.379716 29050 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:26.390466 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:26.393590 29571 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.398273 29571 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.398687 29571 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.402346 29609 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.407686 29200 tablet_bootstrap.cc:654] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.400585 29557 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.416682 29003 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.428022 29571 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.440989 28905 catalog_manager.cc:5697] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.447387 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:26.449246 29048 leader_election.cc:343] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.472728 29200 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:26.474715 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:26.477486 29152 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.474534 29571 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.476859 29199 tablet_bootstrap.cc:654] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.498437 29200 ts_tablet_manager.cc:1403] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.163s	user 0.019s	sys 0.006s
I20260709 13:10:26.499059 29152 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:26.503618 29050 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:26.507445 29567 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.507952 29567 raft_consensus.cc:697] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:26.509090 29567 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.511813 29200 raft_consensus.cc:359] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.512569 29200 raft_consensus.cc:385] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.512995 29200 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.513988 29200 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.523355 29200 ts_tablet_manager.cc:1434] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:10:26.524348 29200 tablet_bootstrap.cc:492] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:26.526093 29199 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:26.526577 29199 ts_tablet_manager.cc:1403] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.201s	user 0.013s	sys 0.000s
I20260709 13:10:26.529947 29200 tablet_bootstrap.cc:654] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.532162 29199 raft_consensus.cc:359] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.532766 29199 raft_consensus.cc:385] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.532944 29199 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.542124 28907 catalog_manager.cc:5697] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.566431 29199 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.568928 29199 ts_tablet_manager.cc:1434] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:10:26.571007 29200 tablet_bootstrap.cc:492] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:26.571444 29200 ts_tablet_manager.cc:1403] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.001s
I20260709 13:10:26.573833 29200 raft_consensus.cc:359] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.574493 29200 raft_consensus.cc:385] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.574753 29200 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.575845 29200 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.579041 29199 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:26.583921 29200 ts_tablet_manager.cc:1434] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:10:26.587343 29199 tablet_bootstrap.cc:654] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.606053 29200 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:26.607461 29199 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:26.607980 29199 ts_tablet_manager.cc:1403] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.029s	user 0.004s	sys 0.011s
I20260709 13:10:26.611049 29199 raft_consensus.cc:359] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.611744 29199 raft_consensus.cc:385] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.612010 29199 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.612943 29200 tablet_bootstrap.cc:654] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.612782 29199 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.615029 29199 ts_tablet_manager.cc:1434] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:26.616387 29199 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:26.623515 29571 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.624487 29200 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:26.624320 29261 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.625020 29200 ts_tablet_manager.cc:1403] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.002s
I20260709 13:10:26.625203 29261 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.627056 29261 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:26.627796 29200 raft_consensus.cc:359] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.628829 29200 raft_consensus.cc:385] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.628856 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:26.629243 29200 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.629568 29003 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:26.630177 29200 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.630769 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:26.631968 29200 ts_tablet_manager.cc:1434] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:26.624017 29571 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.633630 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:26.634244 29084 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 0.
I20260709 13:10:26.634491 29571 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.635972 29261 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.636356 29261 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.636665 29261 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.637194 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.637786 29153 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:26.638861 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:26.633633 29200 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:26.642809 29571 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.643195 29571 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.643528 29571 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.649591 29200 tablet_bootstrap.cc:654] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.657939 29261 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.660020 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:26.660293 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:26.660789 29002 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:26.661262 29048 consensus_peers.cc:597] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.662094 29048 leader_election.cc:343] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.665097 29571 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.667532 29002 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:26.667768 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:26.668653 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:10:26.668890 29048 leader_election.cc:343] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.669443 29602 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.669896 29602 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:26.670256 29261 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:26.670738 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:26.670684 29602 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.672593 29084 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.675266 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:26.676019 29153 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.672021 29199 tablet_bootstrap.cc:654] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.679841 29084 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:26.682691 29200 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:26.683197 29200 ts_tablet_manager.cc:1403] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.011s
I20260709 13:10:26.686168 29571 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.686440 29200 raft_consensus.cc:359] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.687358 29200 raft_consensus.cc:385] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.687656 29200 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.688309 29200 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.689236 29199 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:26.689970 29199 ts_tablet_manager.cc:1403] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.001s
I20260709 13:10:26.690335 29200 ts_tablet_manager.cc:1434] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
W20260709 13:10:26.699448 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:26.686388 29153 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:26.702135 28907 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:26.704507 29200 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: Bootstrap starting.
I20260709 13:10:26.696262 29564 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.705684 29199 raft_consensus.cc:359] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.706336 29199 raft_consensus.cc:385] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.706601 29199 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.707664 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:26.707793 29199 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.709753 29199 ts_tablet_manager.cc:1434] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.000s
I20260709 13:10:26.706446 29564 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.716089 29564 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.713056 29571 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.718576 29571 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:26.719376 29571 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.721480 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.722123 29151 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:26.723663 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:26.725086 29199 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:26.727211 29200 tablet_bootstrap.cc:654] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.728279 29440 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.729225 29440 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.729617 29440 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.729707 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:26.736688 29440 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.738860 29440 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:26.739213 29048 leader_election.cc:343] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.739992 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:26.740769 29152 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.745481 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:26.744024 29199 tablet_bootstrap.cc:654] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.747138 29008 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.747164 28907 catalog_manager.cc:5697] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:26.752705 29048 leader_election.cc:343] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.757341 29199 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:26.757812 29199 ts_tablet_manager.cc:1403] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.001s
I20260709 13:10:26.759792 29567 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.760272 29567 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.762291 29567 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.763213 29199 raft_consensus.cc:359] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.764750 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.765369 29153 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:26.765862 29200 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: No bootstrap required, opened a new log
I20260709 13:10:26.766299 29200 ts_tablet_manager.cc:1403] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: Time spent bootstrapping tablet: real 0.063s	user 0.023s	sys 0.007s
I20260709 13:10:26.763895 29199 raft_consensus.cc:385] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.766700 29199 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.763324 29152 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:26.769594 29199 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.772756 29567 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.773231 29567 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.775189 29567 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.776978 29050 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:26.777662 29564 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.777766 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:26.778038 29564 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.778426 29564 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.778535 29440 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.779246 29440 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:26.779397 29561 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.779853 29561 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.780118 29440 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.781785 29561 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.783587 29200 raft_consensus.cc:359] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.783797 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.784483 29153 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:26.785611 29050 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:26.785837 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:26.786314 29200 raft_consensus.cc:385] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.786625 29200 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.787107 29561 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.787513 29561 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.787288 29200 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.787446 29564 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.787930 29561 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.788589 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.789269 29152 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 0.
I20260709 13:10:26.790009 29564 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.791112 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:26.791769 29151 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.792826 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:26.777766 29199 ts_tablet_manager.cc:1434] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:10:26.794663 29199 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:26.794565 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:26.794842 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:26.796381 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:26.797223 29567 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:26.797585 29567 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.798137 29567 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:26.799257 29048 leader_election.cc:343] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.804822 29561 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.808732 29151 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:26.808656 28907 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.810721 29200 ts_tablet_manager.cc:1434] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:10:26.810695 29567 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.812570 29199 tablet_bootstrap.cc:654] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:26.809442 29048 leader_election.cc:343] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:26.816938 29048 leader_election.cc:343] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:26.817763 29048 leader_election.cc:343] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.817880 29050 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:26.819250 29440 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.820106 29440 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:26.821199 29440 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.823110 29476 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.823731 29476 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.828410 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.829734 29476 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.831645 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:26.844924 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:26.851332 29476 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.853834 29564 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:26.862421 29567 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.867162 29561 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.875629 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:26.876343 29151 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.881897 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:26.884600 28907 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.887372 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:26.887933 29152 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.883292 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:10:26.890129 29048 leader_election.cc:343] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:26.890717 29048 leader_election.cc:343] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:26.900007 29151 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:26.904258 29199 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:26.904729 29199 ts_tablet_manager.cc:1403] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.110s	user 0.038s	sys 0.045s
I20260709 13:10:26.905303 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:26.906105 29385 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.906564 29385 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:26.907380 29385 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.908448 29199 raft_consensus.cc:359] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.909129 29199 raft_consensus.cc:385] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.909372 29199 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.910004 29199 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.911525 29152 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:26.912240 29199 ts_tablet_manager.cc:1434] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:26.913667 29199 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:26.926887 29050 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:26.931171 29567 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:26.932760 29199 tablet_bootstrap.cc:654] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.933116 29567 raft_consensus.cc:697] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:26.934602 29567 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.945979 29199 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:26.946482 29199 ts_tablet_manager.cc:1403] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.001s
I20260709 13:10:26.949112 29199 raft_consensus.cc:359] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.949770 29199 raft_consensus.cc:385] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.950083 29199 raft_consensus.cc:740] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.949869 28905 catalog_manager.cc:5697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.950928 29199 consensus_queue.cc:260] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.953243 29199 ts_tablet_manager.cc:1434] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:10:26.954118 29199 tablet_bootstrap.cc:492] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:26.960390 29199 tablet_bootstrap.cc:654] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.968601 29564 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:26.970299 29199 tablet_bootstrap.cc:492] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:26.970790 29199 ts_tablet_manager.cc:1403] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.017s	user 0.005s	sys 0.008s
I20260709 13:10:26.972308 29476 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.972765 29476 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.974671 29199 raft_consensus.cc:359] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.975455 29199 raft_consensus.cc:385] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:26.975733 29476 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:26.980424 29199 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:26.981215 29199 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:26.983346 29199 ts_tablet_manager.cc:1434] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:10:26.984830 29199 tablet_bootstrap.cc:492] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:26.977543 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:26.977835 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:26.987332 28975 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:26.988332 29476 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:26.990936 29199 tablet_bootstrap.cc:654] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:26.991649 29002 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:26.999938 29567 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:10:27.040413 28905 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.047989 29199 tablet_bootstrap.cc:492] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:27.048537 29199 ts_tablet_manager.cc:1403] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.064s	user 0.005s	sys 0.033s
I20260709 13:10:27.050721 29199 raft_consensus.cc:359] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.051360 29199 raft_consensus.cc:385] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.051606 29199 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.055433 29199 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.057742 29199 ts_tablet_manager.cc:1434] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.004s
I20260709 13:10:27.058789 29199 tablet_bootstrap.cc:492] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:27.064985 29199 tablet_bootstrap.cc:654] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.066689 29002 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.076356 29199 tablet_bootstrap.cc:492] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:27.076830 29199 ts_tablet_manager.cc:1403] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 13:10:27.084147 29199 raft_consensus.cc:359] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.084707 29199 raft_consensus.cc:385] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.084995 29199 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.085680 29199 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.087688 29199 ts_tablet_manager.cc:1434] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.001s
I20260709 13:10:27.089416 29199 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:27.094293 29199 tablet_bootstrap.cc:654] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.098359 29199 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:27.098755 29199 ts_tablet_manager.cc:1403] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:10:27.101205 29199 raft_consensus.cc:359] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.102478 29199 raft_consensus.cc:385] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.102767 29199 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.103360 29199 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.105240 29199 ts_tablet_manager.cc:1434] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:27.106300 29199 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:27.108947 29002 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.111727 29199 tablet_bootstrap.cc:654] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.111941 29476 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.112465 29476 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.114447 29476 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:27.116375 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:27.119361 29567 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.122382 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:27.132254 29152 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:27.134044 28975 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:27.135421 29476 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.141372 29544 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.141824 29544 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.143451 29544 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:27.144174 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:27.145959 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:27.146411 29476 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.146719 29151 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:27.147954 28975 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:27.148744 29476 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.155879 29476 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.161790 29262 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.162199 29199 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:27.162623 29199 ts_tablet_manager.cc:1403] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.057s	user 0.024s	sys 0.028s
I20260709 13:10:27.164628 29567 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.165416 29199 raft_consensus.cc:359] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.166123 29199 raft_consensus.cc:385] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.166298 29002 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.166456 29199 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.167138 29199 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.169193 29199 ts_tablet_manager.cc:1434] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:27.172207 29199 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Bootstrap starting.
I20260709 13:10:27.180068 29199 tablet_bootstrap.cc:654] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:27.182893 29261 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.208618 29544 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.209286 29544 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.213181 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:27.214015 29084 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:27.225473 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:27.226498 29476 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:27.227176 29476 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.227645 29476 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.245360 29544 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:27.248162 29544 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.248716 29544 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.250831 29544 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:27.260991 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:27.264920 29008 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:27.267539 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:27.277638 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:27.278503 29153 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:27.278831 29476 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.279726 28975 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:27.281086 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:27.281317 29476 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:27.281714 29151 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 0.
I20260709 13:10:27.284746 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:27.285876 29153 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.290357 29385 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.291982 29544 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.292829 29385 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.295913 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:27.296730 29080 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.298624 29153 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:27.298976 29199 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: No bootstrap required, opened a new log
I20260709 13:10:27.302667 29199 ts_tablet_manager.cc:1403] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Time spent bootstrapping tablet: real 0.131s	user 0.035s	sys 0.067s
I20260709 13:10:27.304152 28975 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:27.305045 29544 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:27.305552 29544 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:27.305543 29199 raft_consensus.cc:359] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.306371 29199 raft_consensus.cc:385] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:27.306672 29199 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Initialized, Role: FOLLOWER
I20260709 13:10:27.306592 29544 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.307449 29199 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.317265 29080 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:27.357879 29199 ts_tablet_manager.cc:1434] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Time spent starting tablet: real 0.055s	user 0.017s	sys 0.005s
I20260709 13:10:27.361645 28906 catalog_manager.cc:5697] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.369832 29257 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.414710 29257 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.425385 29385 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.458662 29257 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.469321 29385 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.528198 29544 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.528791 29544 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.532241 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:27.532480 29544 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:27.533063 29084 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:27.533385 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:27.534082 29151 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 1: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:27.535267 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:27.536253 29544 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.541425 29544 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:27.624899 29008 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:27.713047 29261 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.713531 29261 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.715410 29261 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:27.716076 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:27.716727 29008 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:27.716794 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:27.717479 29084 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:27.717919 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:27.718935 29261 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:27.719285 29261 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.719604 29261 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:27.726238 29261 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:27.728497 29261 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:27.728906 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:27.729102 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:27.729550 29008 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:27.729558 29084 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:27.735730 29008 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:27.736114 29084 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:27.736713 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:27.737423 29261 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:27.737861 29261 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:27.738595 29261 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:27.746721 28956 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:27.749990 28905 catalog_manager.cc:5697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 0 to 2, leader changed from <none> to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:27.775307 29655 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:27.785372 29544 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:28.189903 29002 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:28.191140 29602 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:28.204068 29080 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:28.205479 29602 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:29.541425 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:29.573668 29671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:29.575052 29674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:29.576227 29672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:29.577999 27901 server_base.cc:1061] running on GCE node
I20260709 13:10:29.579950 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:29.580235 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:29.580449 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602629580413 us; error 0 us; skew 500 ppm
I20260709 13:10:29.581189 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:29.584362 27901 webserver.cc:533] Webserver started at http://127.27.63.68:37767/ using document root <none> and password file <none>
I20260709 13:10:29.584810 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:29.584977 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:29.585233 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:29.586280 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-3-root/instance:
uuid: "ca45420e033847489ad989c1c6eed3ef"
format_stamp: "Formatted at 2026-07-09 13:10:29 on dist-test-slave-sjgx"
I20260709 13:10:29.590837 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:10:29.594007 29679 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:29.594748 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:10:29.595250 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-3-root
uuid: "ca45420e033847489ad989c1c6eed3ef"
format_stamp: "Formatted at 2026-07-09 13:10:29 on dist-test-slave-sjgx"
I20260709 13:10:29.595567 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:29.605574 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:29.607246 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:29.609979 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:29.612756 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:29.613049 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:29.613421 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:29.613641 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:29.665107 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.68:44167
I20260709 13:10:29.665413 29741 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.68:44167 every 8 connection(s)
I20260709 13:10:29.669963 27901 auto_leader_rebalancer-test.cc:384] add a tserver: ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:29.686056 29742 heartbeater.cc:344] Connected to a master server at 127.27.63.126:43467
I20260709 13:10:29.686544 29742 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:29.687378 29742 heartbeater.cc:507] Master 127.27.63.126:43467 requested a full tablet report, sending...
I20260709 13:10:29.689895 28905 ts_manager.cc:194] Registered new tserver with Master: ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:29.691464 28905 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51350
I20260709 13:10:29.879262 29080 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:29.888496 29153 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:29.889211 29002 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:29.889914 29670 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:29.890873 29669 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:29.900704 29051 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d83f8e6d06054712b0c7cd2c5690ec63. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:29.910200 29670 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:29.920454 29153 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:29.922983 29002 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:29.927922 29080 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:29.934999 28905 catalog_manager.cc:5697] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:29.946094 29158 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:10:29.948745 29051 consensus_peers.cc:597] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce7f7e5792294b1f99a1bf4ee332a76c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:29.953125 29753 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:29.958424 29002 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:29.963001 29669 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:29.993005 29753 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:29.993983 29153 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.017693 29002 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.031999 28906 catalog_manager.cc:5697] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.047274 29003 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.065083 29153 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:30.067219 29084 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:30.068252 29655 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:30.071015 29655 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:30.078680 28976 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 49241be91ca743cbac56eb24218ff775. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:30.086019 29476 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.089493 29078 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.091293 29153 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.101523 29008 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.107378 28906 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.126897 29158 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:10:30.127148 28976 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86e87ad9e9a5439d890ebaf60f043eb6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:30.127811 29084 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:30.128208 29771 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:30.129387 29476 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:30.138219 29771 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.139922 29158 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.147557 29084 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:30.159782 28907 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.337229 29775 ts_tablet_manager.cc:933] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:30.339082 29775 tablet_copy_client.cc:323] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:30.357553 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet d83f8e6d06054712b0c7cd2c5690ec63 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:30.358152 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet d83f8e6d06054712b0c7cd2c5690ec63 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-d83f8e6d06054712b0c7cd2c5690ec63
I20260709 13:10:30.365384 29095 tablet_copy_source_session.cc:215] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:30.370083 29775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d83f8e6d06054712b0c7cd2c5690ec63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:30.385668 29775 tablet_copy_client.cc:806] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:30.386322 29775 tablet_copy_client.cc:670] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:30.390353 29775 tablet_copy_client.cc:538] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:30.399541 29775 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:30.424655 29775 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:30.425578 29775 tablet_bootstrap.cc:492] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:30.426214 29775 ts_tablet_manager.cc:1403] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.004s
I20260709 13:10:30.429114 29775 raft_consensus.cc:359] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.429960 29775 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:30.430585 29775 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.434561 29742 heartbeater.cc:499] Master 127.27.63.126:43467 was elected leader, sending a full tablet report...
I20260709 13:10:30.434737 29775 ts_tablet_manager.cc:1434] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.004s
I20260709 13:10:30.437026 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-d83f8e6d06054712b0c7cd2c5690ec63 received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:30.437464 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-d83f8e6d06054712b0c7cd2c5690ec63 on tablet d83f8e6d06054712b0c7cd2c5690ec63 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:30.516431 29775 ts_tablet_manager.cc:933] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:10:30.524075 29775 tablet_copy_client.cc:323] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:10:30.551855 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet 49241be91ca743cbac56eb24218ff775 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:10:30.552598 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet 49241be91ca743cbac56eb24218ff775 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-49241be91ca743cbac56eb24218ff775
I20260709 13:10:30.559823 29020 tablet_copy_source_session.cc:215] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:30.563115 29775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49241be91ca743cbac56eb24218ff775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:30.578490 29782 ts_tablet_manager.cc:933] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:30.582185 29782 tablet_copy_client.cc:323] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:30.583978 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet ce7f7e5792294b1f99a1bf4ee332a76c from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:30.584527 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet ce7f7e5792294b1f99a1bf4ee332a76c from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-ce7f7e5792294b1f99a1bf4ee332a76c
I20260709 13:10:30.584648 29775 tablet_copy_client.cc:806] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:30.585209 29775 tablet_copy_client.cc:670] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:30.589564 29775 tablet_copy_client.cc:538] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:30.591868 29095 tablet_copy_source_session.cc:215] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:30.595105 29782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce7f7e5792294b1f99a1bf4ee332a76c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:30.596894 29775 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:30.610612 29782 tablet_copy_client.cc:806] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:30.611477 29782 tablet_copy_client.cc:670] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:30.615850 29782 tablet_copy_client.cc:538] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:30.617182 29775 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:30.618075 29775 tablet_bootstrap.cc:492] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:30.618741 29775 ts_tablet_manager.cc:1403] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 13:10:30.621716 29775 raft_consensus.cc:359] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.622517 29775 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:30.623126 29775 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.624943 29782 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:30.626605 29775 ts_tablet_manager.cc:1434] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:30.629973 29020 tablet_copy_service.cc:342] P 06ae4197fd784525b695a9b16c8094ea: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-49241be91ca743cbac56eb24218ff775 received from {username='slave'} at 127.0.0.1:33750
I20260709 13:10:30.630457 29020 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-49241be91ca743cbac56eb24218ff775 on tablet 49241be91ca743cbac56eb24218ff775 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:30.644563 29782 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:30.645428 29782 tablet_bootstrap.cc:492] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:30.645988 29782 ts_tablet_manager.cc:1403] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:10:30.648551 29782 raft_consensus.cc:359] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.649336 29782 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:30.649957 29782 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.652175 29782 ts_tablet_manager.cc:1434] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:10:30.654572 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-ce7f7e5792294b1f99a1bf4ee332a76c received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:30.654979 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-ce7f7e5792294b1f99a1bf4ee332a76c on tablet ce7f7e5792294b1f99a1bf4ee332a76c with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:30.675813 29782 ts_tablet_manager.cc:933] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:10:30.681093 29782 tablet_copy_client.cc:323] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:10:30.682832 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet 86e87ad9e9a5439d890ebaf60f043eb6 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:10:30.683303 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet 86e87ad9e9a5439d890ebaf60f043eb6 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-86e87ad9e9a5439d890ebaf60f043eb6
I20260709 13:10:30.688656 29020 tablet_copy_source_session.cc:215] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:30.692664 29782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e87ad9e9a5439d890ebaf60f043eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:30.705859 29782 tablet_copy_client.cc:806] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:30.706534 29782 tablet_copy_client.cc:670] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:30.710469 29782 tablet_copy_client.cc:538] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:30.716307 29782 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:30.733807 29782 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:30.734601 29782 tablet_bootstrap.cc:492] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:30.735226 29782 ts_tablet_manager.cc:1403] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.007s
I20260709 13:10:30.738149 29782 raft_consensus.cc:359] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.738965 29782 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:30.739583 29782 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:30.742136 29782 ts_tablet_manager.cc:1434] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 13:10:30.743744 29020 tablet_copy_service.cc:342] P 06ae4197fd784525b695a9b16c8094ea: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-86e87ad9e9a5439d890ebaf60f043eb6 received from {username='slave'} at 127.0.0.1:33750
I20260709 13:10:30.744163 29020 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-86e87ad9e9a5439d890ebaf60f043eb6 on tablet 86e87ad9e9a5439d890ebaf60f043eb6 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:30.843778 29717 raft_consensus.cc:1217] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:31.023700 29717 raft_consensus.cc:1217] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:31.133620 29717 raft_consensus.cc:1217] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:31.256513 29717 raft_consensus.cc:1217] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:31.330825 29669 raft_consensus.cc:1064] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:31.336814 29669 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:31.354174 29717 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.355963 29008 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.356562 29669 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:31.371934 29669 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:10:31.391084 29153 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.406980 29781 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:31.424269 29753 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.433146 29158 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.436101 29669 raft_consensus.cc:1064] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:31.439181 29669 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:31.451578 28905 catalog_manager.cc:5697] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.431090 29717 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:10:31.487205 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 47029.9us
I20260709 13:10:31.488629 29158 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.490258 29002 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.490942 29800 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:31.490500 29009 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.497968 29717 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.500705 29781 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:31.534636 29789 raft_consensus.cc:1064] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:31.537196 29789 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:31.582861 29799 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:31.587139 29798 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.593108 29717 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.611529 28906 catalog_manager.cc:5697] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.612457 29153 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.617769 29158 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.619184 29771 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:31.624197 29084 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:31.630018 29717 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:31.631599 29800 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:31.638827 29805 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.641546 29716 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.650127 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d83f8e6d06054712b0c7cd2c5690ec63 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:31.667228 28907 catalog_manager.cc:5697] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.669620 29078 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:31.677088 29151 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:31.678671 29716 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:31.680181 29800 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:31.683789 29796 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:31.696398 29717 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.696609 29078 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.698575 29153 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.699772 29803 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.702327 29001 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:31.712613 29771 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:31.715389 29805 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:31.718211 29771 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:31.725049 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet d83f8e6d06054712b0c7cd2c5690ec63 with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35252
I20260709 13:10:31.726131 29717 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.739343 29001 raft_consensus.cc:2955] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.745955 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ce7f7e5792294b1f99a1bf4ee332a76c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:31.747567 29476 raft_consensus.cc:1064] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:31.750100 29476 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:31.761952 28906 catalog_manager.cc:5697] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.763145 29771 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.775082 29717 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.777536 29716 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.780510 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet ce7f7e5792294b1f99a1bf4ee332a76c with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34914
I20260709 13:10:31.781826 29151 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.789397 29789 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:31.790165 29084 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:10:31.797825 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53141.2us
I20260709 13:10:31.798703 29789 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:31.795528 29158 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:31.801127 29820 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:31.806972 28906 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.811324 29816 tablet_replica.cc:333] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:10:31.813977 29816 raft_consensus.cc:2243] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.817126 29816 raft_consensus.cc:2272] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.835137 29009 raft_consensus.cc:2955] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.841032 29816 ts_tablet_manager.cc:1916] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:31.840179 29820 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.845914 29151 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.854173 29716 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.868136 29821 tablet_replica.cc:333] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:10:31.869282 29821 raft_consensus.cc:2243] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.869935 29821 raft_consensus.cc:2272] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.873205 29821 ts_tablet_manager.cc:1916] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:31.877307 29078 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.884222 28906 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.883066 29084 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.920306 29821 ts_tablet_manager.cc:1929] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:31.920873 29821 log.cc:1199] T ce7f7e5792294b1f99a1bf4ee332a76c P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/ce7f7e5792294b1f99a1bf4ee332a76c
I20260709 13:10:31.921061 29009 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:31.928524 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet ce7f7e5792294b1f99a1bf4ee332a76c (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:31.935043 29816 ts_tablet_manager.cc:1929] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:31.935837 29816 log.cc:1199] T d83f8e6d06054712b0c7cd2c5690ec63 P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/d83f8e6d06054712b0c7cd2c5690ec63
I20260709 13:10:31.938761 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet d83f8e6d06054712b0c7cd2c5690ec63 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:31.941445 29002 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:31.949805 29716 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:31.950613 29151 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:31.952955 29716 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:31.953243 29820 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:31.953747 29151 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:31.954980 29820 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:31.957574 29813 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:31.959362 29813 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:31.976339 29789 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.981392 29476 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.986020 29153 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:31.990994 29151 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:32.019853 28907 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:32.038303 28892 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 49241be91ca743cbac56eb24218ff775 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:32.039211 28892 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 86e87ad9e9a5439d890ebaf60f043eb6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:32.042609 28906 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:32.046254 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet 49241be91ca743cbac56eb24218ff775 with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44418
I20260709 13:10:32.052459 29716 raft_consensus.cc:2955] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:32.056020 29074 tablet_service.cc:1558] Processing DeleteTablet for tablet 86e87ad9e9a5439d890ebaf60f043eb6 with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44418
I20260709 13:10:32.058241 29828 tablet_replica.cc:333] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:10:32.057473 29717 raft_consensus.cc:2955] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:32.059418 29828 raft_consensus.cc:2243] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:32.060267 29828 raft_consensus.cc:2272] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:32.065711 29828 ts_tablet_manager.cc:1916] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:32.084442 29828 ts_tablet_manager.cc:1929] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:32.084842 29828 log.cc:1199] T 49241be91ca743cbac56eb24218ff775 P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/49241be91ca743cbac56eb24218ff775
I20260709 13:10:32.086647 29828 tablet_replica.cc:333] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:10:32.087123 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet 49241be91ca743cbac56eb24218ff775 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:32.087826 29828 raft_consensus.cc:2243] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:32.088510 29828 raft_consensus.cc:2272] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:32.092180 29828 ts_tablet_manager.cc:1916] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:32.103206 29828 ts_tablet_manager.cc:1929] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:32.103606 29828 log.cc:1199] T 86e87ad9e9a5439d890ebaf60f043eb6 P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/86e87ad9e9a5439d890ebaf60f043eb6
I20260709 13:10:32.106060 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet 86e87ad9e9a5439d890ebaf60f043eb6 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:33.135177 29009 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.142448 29153 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:10:33.142671 28976 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21d163c72e704bd0bd37c74d96761663. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.143006 29078 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:33.143997 29825 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:33.145119 29826 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:33.153501 29825 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.155081 29078 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.157362 29153 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.200574 28906 catalog_manager.cc:5697] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.226491 29153 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.235494 29009 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:33.237370 29791 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:33.248086 29839 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.250433 29009 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.256003 29078 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:33.262280 29838 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:33.276939 29084 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.281692 28908 catalog_manager.cc:5697] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:33.287422 29051 consensus_peers.cc:597] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94045d5a8709418e82919c9fbf804582. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.294163 29009 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:33.295513 29078 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.296681 29836 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:33.300694 29153 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:10:33.306031 29125 consensus_peers.cc:597] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 29895dd2e2b6460b8106269935db24f7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.309417 29796 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:33.315790 29836 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.326074 29009 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.336863 29078 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.363966 28908 catalog_manager.cc:5697] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.368570 29151 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
W20260709 13:10:33.387687 29051 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dafcee70c76341f2a36a461c97f2b3ba. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:33.396289 29151 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:33.397240 29009 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:33.398397 29803 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:33.400856 29796 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:33.413188 29853 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.414824 29009 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.416713 29151 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:33.428000 28908 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.586635 29857 ts_tablet_manager.cc:933] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:10:33.589061 29857 tablet_copy_client.cc:323] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:10:33.591274 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet 21d163c72e704bd0bd37c74d96761663 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:10:33.591888 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet 21d163c72e704bd0bd37c74d96761663 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-21d163c72e704bd0bd37c74d96761663
I20260709 13:10:33.599769 29020 tablet_copy_source_session.cc:215] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:33.602730 29857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d163c72e704bd0bd37c74d96761663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:33.618351 29857 tablet_copy_client.cc:806] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:33.619025 29857 tablet_copy_client.cc:670] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:33.622881 29857 tablet_copy_client.cc:538] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:33.630288 29857 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:33.662719 29857 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.663836 29857 tablet_bootstrap.cc:492] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:33.664578 29857 ts_tablet_manager.cc:1403] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.035s	user 0.028s	sys 0.006s
I20260709 13:10:33.667738 29857 raft_consensus.cc:359] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.668656 29857 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:33.669294 29857 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.674644 29857 ts_tablet_manager.cc:1434] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:10:33.676384 29020 tablet_copy_service.cc:342] P 06ae4197fd784525b695a9b16c8094ea: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-21d163c72e704bd0bd37c74d96761663 received from {username='slave'} at 127.0.0.1:33750
I20260709 13:10:33.676872 29020 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-21d163c72e704bd0bd37c74d96761663 on tablet 21d163c72e704bd0bd37c74d96761663 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:33.711673 29857 ts_tablet_manager.cc:933] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:33.714224 29857 tablet_copy_client.cc:323] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:33.716681 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 94045d5a8709418e82919c9fbf804582 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:33.717154 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 94045d5a8709418e82919c9fbf804582 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-94045d5a8709418e82919c9fbf804582
I20260709 13:10:33.724503 29095 tablet_copy_source_session.cc:215] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:33.727265 29862 ts_tablet_manager.cc:933] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:33.728430 29857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94045d5a8709418e82919c9fbf804582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:33.740788 29862 tablet_copy_client.cc:323] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:10:33.745956 29857 tablet_copy_client.cc:806] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:33.746630 29857 tablet_copy_client.cc:670] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:33.753620 29857 tablet_copy_client.cc:538] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:33.763517 29857 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:33.784379 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet 29895dd2e2b6460b8106269935db24f7 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:10:33.784991 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet 29895dd2e2b6460b8106269935db24f7 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-29895dd2e2b6460b8106269935db24f7
I20260709 13:10:33.790541 29857 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.791476 29857 tablet_bootstrap.cc:492] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:33.792207 29857 ts_tablet_manager.cc:1403] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.029s	user 0.026s	sys 0.000s
I20260709 13:10:33.797613 29169 tablet_copy_source_session.cc:215] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:33.797863 29857 raft_consensus.cc:359] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.801062 29862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29895dd2e2b6460b8106269935db24f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:33.799032 29857 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:33.804629 29857 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.807178 29857 ts_tablet_manager.cc:1434] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:10:33.809253 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-94045d5a8709418e82919c9fbf804582 received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:33.809664 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-94045d5a8709418e82919c9fbf804582 on tablet 94045d5a8709418e82919c9fbf804582 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:33.820500 29862 tablet_copy_client.cc:806] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:33.821091 29862 tablet_copy_client.cc:670] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:33.824608 29862 tablet_copy_client.cc:538] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:33.832804 29862 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:33.852087 29862 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.852982 29862 tablet_bootstrap.cc:492] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:33.853611 29862 ts_tablet_manager.cc:1403] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:10:33.856530 29862 raft_consensus.cc:359] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.857501 29862 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:33.858197 29862 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.862720 29862 ts_tablet_manager.cc:1434] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:33.864588 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-29895dd2e2b6460b8106269935db24f7 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:10:33.865090 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-29895dd2e2b6460b8106269935db24f7 on tablet 29895dd2e2b6460b8106269935db24f7 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:33.888279 29862 ts_tablet_manager.cc:933] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:33.893613 29862 tablet_copy_client.cc:323] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:33.894969 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet dafcee70c76341f2a36a461c97f2b3ba from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:33.895479 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet dafcee70c76341f2a36a461c97f2b3ba from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-dafcee70c76341f2a36a461c97f2b3ba
I20260709 13:10:33.903143 29095 tablet_copy_source_session.cc:215] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:33.906334 29862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafcee70c76341f2a36a461c97f2b3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:33.922061 29862 tablet_copy_client.cc:806] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:33.922709 29862 tablet_copy_client.cc:670] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:33.927453 29862 tablet_copy_client.cc:538] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:33.938227 29862 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:33.958254 29862 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:33.959219 29862 tablet_bootstrap.cc:492] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:33.959905 29862 ts_tablet_manager.cc:1403] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.004s
I20260709 13:10:33.962831 29862 raft_consensus.cc:359] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.963752 29862 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:33.964461 29862 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:33.966810 29862 ts_tablet_manager.cc:1434] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:33.968668 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-dafcee70c76341f2a36a461c97f2b3ba received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:33.969080 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-dafcee70c76341f2a36a461c97f2b3ba on tablet dafcee70c76341f2a36a461c97f2b3ba with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:34.043812 29717 raft_consensus.cc:1217] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:34.121155 29717 raft_consensus.cc:1217] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:34.277685 29717 raft_consensus.cc:1217] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:34.388463 29717 raft_consensus.cc:1217] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:34.468647 29833 raft_consensus.cc:1064] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:34.472517 29833 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:34.490844 29151 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.492242 29717 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.493558 29870 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:10:34.496446 29871 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:34.502107 29084 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.503821 29833 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:34.524938 29836 raft_consensus.cc:1064] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:34.528307 29836 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:34.536644 29871 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.546873 29717 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.547477 29151 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.549841 29716 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.551676 29009 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.556092 29151 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.564388 28907 catalog_manager.cc:5697] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.548468 29836 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:34.588192 29854 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:34.589862 29872 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:34.629299 29078 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.645431 29872 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.650713 29151 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.662997 29716 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.686682 29009 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.721431 28906 catalog_manager.cc:5697] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.728216 29002 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:34.736259 29078 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:34.737996 29716 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:34.745129 29874 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:34.745426 29717 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:34.749954 29153 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:34.752357 29853 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:34.754889 29872 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:34.757596 29786 raft_consensus.cc:1064] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:34.762810 29078 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:34.762238 29874 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.759963 29786 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:34.771055 29884 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:34.775024 29717 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.787950 29078 raft_consensus.cc:2955] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.803542 28906 catalog_manager.cc:5697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.791801 29880 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.797164 29715 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.814747 28892 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 21d163c72e704bd0bd37c74d96761663 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:34.800920 29714 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.834622 29153 raft_consensus.cc:2955] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:10:34.851861 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44442.3us
I20260709 13:10:34.854004 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 94045d5a8709418e82919c9fbf804582 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:10:34.858093 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56175.6us
I20260709 13:10:34.860394 29078 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.862850 29009 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.877250 28908 catalog_manager.cc:5697] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.871253 29893 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:34.880239 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet 21d163c72e704bd0bd37c74d96761663 with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35252
I20260709 13:10:34.882485 29839 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:34.884575 29839 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:34.891948 29894 tablet_replica.cc:333] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:10:34.894421 29894 raft_consensus.cc:2243] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.895251 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 94045d5a8709418e82919c9fbf804582 with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34914
I20260709 13:10:34.895242 29894 raft_consensus.cc:2272] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.899760 29894 ts_tablet_manager.cc:1916] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:34.922147 29894 ts_tablet_manager.cc:1929] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:34.924527 29894 log.cc:1199] T 21d163c72e704bd0bd37c74d96761663 P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/21d163c72e704bd0bd37c74d96761663
I20260709 13:10:34.927249 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet 21d163c72e704bd0bd37c74d96761663 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:34.936440 29897 tablet_replica.cc:333] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:10:34.939764 29839 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.943130 29897 raft_consensus.cc:2243] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.944065 29897 raft_consensus.cc:2272] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.945807 29715 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.952950 29897 ts_tablet_manager.cc:1916] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:34.953464 29853 raft_consensus.cc:1064] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:34.959115 29853 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:34.948014 29084 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.967617 28906 catalog_manager.cc:5697] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.968979 29001 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:34.984326 29158 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:34.996665 29150 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:34.997189 29078 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:35.001096 29893 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:35.005857 29716 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:35.010174 29838 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:35.019917 29839 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.020706 29880 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:35.024173 29715 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:35.025976 29891 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:35.025213 29079 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.035144 29001 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:35.037467 29891 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:35.047751 29716 raft_consensus.cc:2955] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.064087 28906 catalog_manager.cc:5697] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.067715 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 29895dd2e2b6460b8106269935db24f7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:35.076115 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 29895dd2e2b6460b8106269935db24f7 with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34914
I20260709 13:10:35.082163 29897 ts_tablet_manager.cc:1929] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:35.085734 29891 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.094265 29150 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.096941 29897 log.cc:1199] T 94045d5a8709418e82919c9fbf804582 P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/94045d5a8709418e82919c9fbf804582
I20260709 13:10:35.101138 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 94045d5a8709418e82919c9fbf804582 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:35.102139 29897 tablet_replica.cc:333] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:10:35.103261 29897 raft_consensus.cc:2243] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:35.103955 29897 raft_consensus.cc:2272] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:35.105723 29715 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.111632 29897 ts_tablet_manager.cc:1916] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:35.116546 29001 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.122907 28906 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.139233 29897 ts_tablet_manager.cc:1929] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:35.139816 29897 log.cc:1199] T 29895dd2e2b6460b8106269935db24f7 P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/29895dd2e2b6460b8106269935db24f7
I20260709 13:10:35.141443 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 29895dd2e2b6460b8106269935db24f7 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:35.160571 29079 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:35.171898 29715 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:35.172600 29001 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:35.173365 29853 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:35.174291 29880 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:35.180110 29880 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.188300 29716 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.188536 29001 raft_consensus.cc:2955] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:35.193388 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dafcee70c76341f2a36a461c97f2b3ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:35.203291 28908 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:35.226135 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet dafcee70c76341f2a36a461c97f2b3ba with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35252
I20260709 13:10:35.227064 29894 tablet_replica.cc:333] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:10:35.228102 29894 raft_consensus.cc:2243] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:35.228825 29894 raft_consensus.cc:2272] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:35.235314 29894 ts_tablet_manager.cc:1916] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:35.247035 29894 ts_tablet_manager.cc:1929] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:35.247359 29894 log.cc:1199] T dafcee70c76341f2a36a461c97f2b3ba P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/dafcee70c76341f2a36a461c97f2b3ba
I20260709 13:10:35.248544 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet dafcee70c76341f2a36a461c97f2b3ba (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:36.293926 29150 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:10:36.300439 29125 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ddba9f7f2a6843c480dac4ac4efa4931. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:36.300732 29079 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:36.302135 29904 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.303562 29009 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:36.304800 29839 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.314517 29904 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.315255 29079 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.316617 29001 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.338480 28907 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.340198 29079 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:36.350100 29009 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:36.352496 29150 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:36.353223 29880 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:36.354496 29051 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 57dd7e7308fc43e0b0983412129aa471. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:36.356910 29880 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.364409 29891 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.368858 29001 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.383924 29158 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.394495 29001 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:36.409027 29150 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:36.410727 29884 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:36.414031 28976 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c32aba72953748b4bb7a95efc76501ff. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:36.414980 28906 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.432452 29079 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:36.438709 29874 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:36.457572 29884 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.460871 29150 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.464746 29079 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.475869 29084 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:36.479350 28907 catalog_manager.cc:5697] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:36.489315 29051 consensus_peers.cc:597] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 68c287f2818e4a23b258f6c19741c26a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:36.493885 29150 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:36.495479 29917 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.497048 29009 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:36.498507 29917 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:36.505175 29915 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.510661 29150 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.521603 28906 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.528575 29009 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:36.784611 29924 ts_tablet_manager.cc:933] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:36.789726 29924 tablet_copy_client.cc:323] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:10:36.791289 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet ddba9f7f2a6843c480dac4ac4efa4931 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:10:36.791787 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet ddba9f7f2a6843c480dac4ac4efa4931 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-ddba9f7f2a6843c480dac4ac4efa4931
I20260709 13:10:36.798211 29169 tablet_copy_source_session.cc:215] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:36.801859 29924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddba9f7f2a6843c480dac4ac4efa4931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:36.822383 29924 tablet_copy_client.cc:806] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:36.823024 29924 tablet_copy_client.cc:670] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:36.826288 29924 tablet_copy_client.cc:538] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:36.835086 29924 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:36.841046 29926 ts_tablet_manager.cc:933] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:10:36.843132 29926 tablet_copy_client.cc:323] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:10:36.844926 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet c32aba72953748b4bb7a95efc76501ff from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:10:36.845469 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet c32aba72953748b4bb7a95efc76501ff from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-c32aba72953748b4bb7a95efc76501ff
I20260709 13:10:36.854954 29924 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:36.855854 29924 tablet_bootstrap.cc:492] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:36.856510 29924 ts_tablet_manager.cc:1403] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.022s	user 0.023s	sys 0.000s
I20260709 13:10:36.860134 29020 tablet_copy_source_session.cc:215] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:36.859922 29924 raft_consensus.cc:359] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:36.860996 29924 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:36.861719 29924 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:36.871459 29924 ts_tablet_manager.cc:1434] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.002s
I20260709 13:10:36.872660 29926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c32aba72953748b4bb7a95efc76501ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:36.875929 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-ddba9f7f2a6843c480dac4ac4efa4931 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:10:36.876403 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-ddba9f7f2a6843c480dac4ac4efa4931 on tablet ddba9f7f2a6843c480dac4ac4efa4931 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:36.892025 29926 tablet_copy_client.cc:806] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:36.892720 29926 tablet_copy_client.cc:670] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:36.896482 29926 tablet_copy_client.cc:538] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:36.904636 29926 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:36.920737 29926 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:36.921622 29926 tablet_bootstrap.cc:492] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:36.922297 29926 ts_tablet_manager.cc:1403] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 13:10:36.925215 29926 raft_consensus.cc:359] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:36.926402 29926 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:36.927121 29926 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:36.929972 29926 ts_tablet_manager.cc:1434] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:36.931721 29020 tablet_copy_service.cc:342] P 06ae4197fd784525b695a9b16c8094ea: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-c32aba72953748b4bb7a95efc76501ff received from {username='slave'} at 127.0.0.1:33750
I20260709 13:10:36.932165 29020 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-c32aba72953748b4bb7a95efc76501ff on tablet c32aba72953748b4bb7a95efc76501ff with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:36.953431 29926 ts_tablet_manager.cc:933] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:36.958515 29926 tablet_copy_client.cc:323] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:36.960093 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 68c287f2818e4a23b258f6c19741c26a from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:36.960630 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 68c287f2818e4a23b258f6c19741c26a from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-68c287f2818e4a23b258f6c19741c26a
I20260709 13:10:36.965473 29924 ts_tablet_manager.cc:933] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:36.967906 29095 tablet_copy_source_session.cc:215] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:36.974929 29926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68c287f2818e4a23b258f6c19741c26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:36.976999 29924 tablet_copy_client.cc:323] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:36.980279 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 57dd7e7308fc43e0b0983412129aa471 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:36.981669 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 57dd7e7308fc43e0b0983412129aa471 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-57dd7e7308fc43e0b0983412129aa471
I20260709 13:10:36.987711 29095 tablet_copy_source_session.cc:215] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:36.990298 29924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dd7e7308fc43e0b0983412129aa471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:37.002264 29926 tablet_copy_client.cc:806] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:37.002897 29926 tablet_copy_client.cc:670] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:37.005159 29924 tablet_copy_client.cc:806] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:37.005705 29924 tablet_copy_client.cc:670] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:37.007138 29926 tablet_copy_client.cc:538] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:37.009122 29924 tablet_copy_client.cc:538] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:37.016072 29926 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:37.018586 29924 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:37.038996 29926 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:37.040565 29926 tablet_bootstrap.cc:492] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:37.041430 29926 ts_tablet_manager.cc:1403] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.026s	user 0.023s	sys 0.005s
I20260709 13:10:37.044651 29926 raft_consensus.cc:359] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:37.045539 29926 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:37.046339 29926 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:37.053553 29926 ts_tablet_manager.cc:1434] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:10:37.060313 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-68c287f2818e4a23b258f6c19741c26a received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:37.060746 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-68c287f2818e4a23b258f6c19741c26a on tablet 68c287f2818e4a23b258f6c19741c26a with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:37.069123 29924 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:37.070155 29924 tablet_bootstrap.cc:492] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:37.070858 29924 ts_tablet_manager.cc:1403] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.053s	user 0.021s	sys 0.008s
I20260709 13:10:37.073807 29924 raft_consensus.cc:359] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:37.074641 29924 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:37.075240 29924 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:37.077596 29924 ts_tablet_manager.cc:1434] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:37.079159 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-57dd7e7308fc43e0b0983412129aa471 received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:37.079543 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-57dd7e7308fc43e0b0983412129aa471 on tablet 57dd7e7308fc43e0b0983412129aa471 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:37.190855 29715 raft_consensus.cc:1217] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:37.397039 29715 raft_consensus.cc:1217] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:37.442868 29715 raft_consensus.cc:1217] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:37.504242 29715 raft_consensus.cc:1217] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:37.768193 29904 raft_consensus.cc:1064] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:37.770684 29904 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:37.778738 29715 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:37.781136 29079 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:37.782085 29002 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:37.781149 29904 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:10:37.784796 29839 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:37.785583 29936 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:37.812909 29936 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.816809 29084 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.824597 29002 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.832846 29715 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.833171 28906 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.876168 29151 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:37.885147 29001 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:37.889534 29936 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:37.889822 29715 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:37.891146 29839 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:37.896030 29936 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.904695 29001 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.905941 29938 raft_consensus.cc:1064] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:37.911525 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ddba9f7f2a6843c480dac4ac4efa4931 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:37.912432 29716 raft_consensus.cc:2955] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.912998 29938 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:37.934455 29909 raft_consensus.cc:1064] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:37.943033 28907 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:37.977633 29716 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:37.979928 29917 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:37.938899 29909 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:37.982235 29151 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:37.987202 29938 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:37.994588 29716 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:37.995792 29150 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:37.998255 29951 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:38.000202 29933 raft_consensus.cc:1064] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:38.003888 29001 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:38.004730 29009 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:38.007377 29933 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:38.008374 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet ddba9f7f2a6843c480dac4ac4efa4931 with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44418
I20260709 13:10:38.014549 29955 tablet_replica.cc:333] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:10:38.014560 29909 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:38.011801 29950 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.018411 29151 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.035679 29955 raft_consensus.cc:2243] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:38.059603 29938 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:38.060550 29955 raft_consensus.cc:2272] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:38.045737 29150 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:38.063836 29933 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:38.047053 29716 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:38.048034 29078 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:38.049438 29949 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:38.047268 28908 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.051721 29153 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.039201 29909 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.072966 29933 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:38.083196 29715 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.086563 29933 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:38.093900 29955 ts_tablet_manager.cc:1916] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:38.093586 29714 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.104439 29078 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:38.110291 29153 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:38.112396 29948 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:38.120684 28907 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.120760 29949 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.128444 29153 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.145500 29717 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:38.150470 29009 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.172478 29950 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:38.172254 28907 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.184239 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 68c287f2818e4a23b258f6c19741c26a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:38.184746 29717 raft_consensus.cc:2955] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.190888 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 68c287f2818e4a23b258f6c19741c26a with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:34914
I20260709 13:10:38.190827 29003 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.208619 29964 tablet_replica.cc:333] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:10:38.217159 29933 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.221028 29964 raft_consensus.cc:2243] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:38.221719 29964 raft_consensus.cc:2272] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:38.225775 29964 ts_tablet_manager.cc:1916] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:38.231782 29955 ts_tablet_manager.cc:1929] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:38.232643 29955 log.cc:1199] T ddba9f7f2a6843c480dac4ac4efa4931 P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/ddba9f7f2a6843c480dac4ac4efa4931
I20260709 13:10:38.227300 29150 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.236574 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet ddba9f7f2a6843c480dac4ac4efa4931 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:38.246178 29717 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.262041 29964 ts_tablet_manager.cc:1929] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:38.262557 29964 log.cc:1199] T 68c287f2818e4a23b258f6c19741c26a P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/68c287f2818e4a23b258f6c19741c26a
I20260709 13:10:38.265488 29078 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.268891 28908 catalog_manager.cc:5697] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.272552 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 68c287f2818e4a23b258f6c19741c26a (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:38.293113 29003 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:38.297782 29150 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:38.297889 29717 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:38.298990 29937 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:38.300674 29966 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:38.305899 29937 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.312784 29717 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.317015 28892 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c32aba72953748b4bb7a95efc76501ff with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:38.318225 29150 raft_consensus.cc:2955] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.330727 28906 catalog_manager.cc:5697] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.345635 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet c32aba72953748b4bb7a95efc76501ff with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44418
I20260709 13:10:38.346514 29955 tablet_replica.cc:333] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:10:38.347254 29955 raft_consensus.cc:2243] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:38.347946 29955 raft_consensus.cc:2272] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:38.351359 29955 ts_tablet_manager.cc:1916] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:38.366108 29955 ts_tablet_manager.cc:1929] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:38.370537 29955 log.cc:1199] T c32aba72953748b4bb7a95efc76501ff P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/c32aba72953748b4bb7a95efc76501ff
I20260709 13:10:38.372537 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet c32aba72953748b4bb7a95efc76501ff (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:38.606643 29078 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:38.612084 29717 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:38.612643 29003 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:38.613344 29915 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:10:38.615634 29951 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:38.625280 29951 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.631491 29716 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.642148 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 57dd7e7308fc43e0b0983412129aa471 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:38.649161 28907 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:38.667176 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet 57dd7e7308fc43e0b0983412129aa471 with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35252
I20260709 13:10:38.669648 29967 tablet_replica.cc:333] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:10:38.665143 29003 raft_consensus.cc:2955] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:38.685346 29967 raft_consensus.cc:2243] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:38.686129 29967 raft_consensus.cc:2272] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:38.688926 29967 ts_tablet_manager.cc:1916] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:38.705031 29967 ts_tablet_manager.cc:1929] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:38.705410 29967 log.cc:1199] T 57dd7e7308fc43e0b0983412129aa471 P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/57dd7e7308fc43e0b0983412129aa471
I20260709 13:10:38.707124 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet 57dd7e7308fc43e0b0983412129aa471 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
W20260709 13:10:39.862947 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 232579us
I20260709 13:10:39.869956 29079 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:10:39.945019 29051 consensus_peers.cc:597] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6415e058581b4b7eb883c6e96b9fbfcf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:39.949306 29003 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:39.950896 29972 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:39.951201 29151 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:39.959301 29972 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:39.978338 29973 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:39.982105 29153 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:39.985894 29001 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:39.994933 29152 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.009421 28908 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:40.033401 29125 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eb070bde5f3744eb90ee5cf3dec8fcd4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:40.035157 29079 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:40.035714 29001 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:40.037051 29968 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.040853 29968 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.070585 29003 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.071137 29968 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.108384 29079 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.116156 28906 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.121940 29009 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.143009 29152 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:40.144687 29933 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.145140 29079 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:40.146517 29980 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:40.149158 28976 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da4f9ce63d54a63a8b9de11ba485699. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:40.158324 29933 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.161222 29153 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.164760 29079 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.175258 29152 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.177197 28907 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
W20260709 13:10:40.199733 29125 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3aa255ba50b343eca7717515ec37d959. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:40.202335 29001 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:40.202335 29079 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:40.204007 29975 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.208632 29975 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.227414 29968 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.230469 29009 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.232033 29079 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.248456 28906 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:40.421087 29996 ts_tablet_manager.cc:933] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:40.425431 29996 tablet_copy_client.cc:323] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:10:40.426959 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet eb070bde5f3744eb90ee5cf3dec8fcd4 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:10:40.427505 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet eb070bde5f3744eb90ee5cf3dec8fcd4 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-eb070bde5f3744eb90ee5cf3dec8fcd4
I20260709 13:10:40.434927 29169 tablet_copy_source_session.cc:215] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:40.438470 29996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb070bde5f3744eb90ee5cf3dec8fcd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:40.453528 29996 tablet_copy_client.cc:806] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:40.454169 29996 tablet_copy_client.cc:670] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:40.457616 29996 tablet_copy_client.cc:538] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:40.464570 29996 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:40.493348 29996 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:40.494247 29996 tablet_bootstrap.cc:492] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:40.494856 29996 ts_tablet_manager.cc:1403] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.008s
I20260709 13:10:40.497493 29996 raft_consensus.cc:359] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.498317 29996 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:40.499030 29996 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.505317 29996 ts_tablet_manager.cc:1434] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:40.506873 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-eb070bde5f3744eb90ee5cf3dec8fcd4 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:10:40.507234 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-eb070bde5f3744eb90ee5cf3dec8fcd4 on tablet eb070bde5f3744eb90ee5cf3dec8fcd4 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:40.523175 29996 ts_tablet_manager.cc:933] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:40.524816 29996 tablet_copy_client.cc:323] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:40.526615 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 6415e058581b4b7eb883c6e96b9fbfcf from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:40.527138 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 6415e058581b4b7eb883c6e96b9fbfcf from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-6415e058581b4b7eb883c6e96b9fbfcf
I20260709 13:10:40.533574 29095 tablet_copy_source_session.cc:215] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:40.542346 29996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6415e058581b4b7eb883c6e96b9fbfcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:40.556715 29996 tablet_copy_client.cc:806] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:40.557284 29996 tablet_copy_client.cc:670] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:40.561121 29996 tablet_copy_client.cc:538] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:40.569872 29996 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:40.584834 29996 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:40.585873 29996 tablet_bootstrap.cc:492] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:40.587004 29996 ts_tablet_manager.cc:1403] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.001s
I20260709 13:10:40.589527 29996 raft_consensus.cc:359] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.590126 29996 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:40.590631 29996 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.592478 29996 ts_tablet_manager.cc:1434] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:10:40.594461 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-6415e058581b4b7eb883c6e96b9fbfcf received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:40.594833 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-6415e058581b4b7eb883c6e96b9fbfcf on tablet 6415e058581b4b7eb883c6e96b9fbfcf with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:40.666818 29996 ts_tablet_manager.cc:933] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:10:40.674687 29996 tablet_copy_client.cc:323] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:10:40.676461 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet 8da4f9ce63d54a63a8b9de11ba485699 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:10:40.677035 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet 8da4f9ce63d54a63a8b9de11ba485699 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-8da4f9ce63d54a63a8b9de11ba485699
I20260709 13:10:40.685190 29020 tablet_copy_source_session.cc:215] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:40.688292 29996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da4f9ce63d54a63a8b9de11ba485699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:40.698879 29996 tablet_copy_client.cc:806] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:40.699364 29996 tablet_copy_client.cc:670] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:40.702991 29996 tablet_copy_client.cc:538] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:40.708189 29996 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:40.724221 29996 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:40.725121 29996 tablet_bootstrap.cc:492] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:40.725769 29996 ts_tablet_manager.cc:1403] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.018s	user 0.019s	sys 0.000s
I20260709 13:10:40.729198 30004 ts_tablet_manager.cc:933] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:40.728614 29996 raft_consensus.cc:359] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.729583 29996 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:40.730252 29996 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.732559 29996 ts_tablet_manager.cc:1434] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:10:40.732784 30004 tablet_copy_client.cc:323] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:10:40.734258 29020 tablet_copy_service.cc:342] P 06ae4197fd784525b695a9b16c8094ea: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-8da4f9ce63d54a63a8b9de11ba485699 received from {username='slave'} at 127.0.0.1:33750
I20260709 13:10:40.734408 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet 3aa255ba50b343eca7717515ec37d959 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:10:40.734692 29020 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-8da4f9ce63d54a63a8b9de11ba485699 on tablet 8da4f9ce63d54a63a8b9de11ba485699 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:40.734997 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet 3aa255ba50b343eca7717515ec37d959 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-3aa255ba50b343eca7717515ec37d959
I20260709 13:10:40.748909 29169 tablet_copy_source_session.cc:215] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:40.751890 30004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aa255ba50b343eca7717515ec37d959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:40.766059 30004 tablet_copy_client.cc:806] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:40.766602 30004 tablet_copy_client.cc:670] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:40.770851 30004 tablet_copy_client.cc:538] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:40.777207 30004 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
W20260709 13:10:40.794077 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7380 after lost signal to thread 27904
W20260709 13:10:40.795081 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7a60 after lost signal to thread 28942
W20260709 13:10:40.796269 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8760 after lost signal to thread 29034
W20260709 13:10:40.797297 29738 debug-util.cc:398] Leaking SignalData structure 0x7b080031b400 after lost signal to thread 29109
W20260709 13:10:40.798251 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002bb5c0 after lost signal to thread 29183
I20260709 13:10:40.798364 30004 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:10:40.799113 29738 debug-util.cc:398] Leaking SignalData structure 0x7b080022d820 after lost signal to thread 29741
I20260709 13:10:40.799285 30004 tablet_bootstrap.cc:492] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:40.799975 30004 ts_tablet_manager.cc:1403] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.004s
I20260709 13:10:40.809432 30004 raft_consensus.cc:359] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.810326 30004 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:40.810971 30004 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:40.813314 30004 ts_tablet_manager.cc:1434] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:40.817647 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-3aa255ba50b343eca7717515ec37d959 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:10:40.818061 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-3aa255ba50b343eca7717515ec37d959 on tablet 3aa255ba50b343eca7717515ec37d959 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:41.011565 29716 raft_consensus.cc:1217] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:41.153692 29716 raft_consensus.cc:1217] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:41.196015 29716 raft_consensus.cc:1217] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:41.204165 29717 raft_consensus.cc:1217] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:41.433012 29975 raft_consensus.cc:1064] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:41.466666 29975 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:41.482272 29717 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.487062 29970 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.490978 29079 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.492724 30008 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.503594 29001 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.505126 30008 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:41.528882 29970 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.530378 29009 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.551617 29717 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.589078 29083 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.555578 28907 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:41.668903 29153 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:41.676867 29714 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:41.677866 29009 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:41.679750 29970 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:41.680948 30008 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:41.688776 29980 raft_consensus.cc:1064] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:41.691291 29975 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.692906 29714 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.693789 29980 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:41.702565 29003 raft_consensus.cc:2955] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.711726 29970 raft_consensus.cc:1064] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:41.714243 29970 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:41.746870 29915 raft_consensus.cc:1064] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:41.749393 29915 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:41.727241 28906 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.800590 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eb070bde5f3744eb90ee5cf3dec8fcd4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:41.819736 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet eb070bde5f3744eb90ee5cf3dec8fcd4 with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44418
I20260709 13:10:41.825021 30021 tablet_replica.cc:333] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:10:41.849821 29714 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.853849 29083 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.859023 30008 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.861018 30008 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.891350 29002 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.892874 29975 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.893373 29151 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.903750 29972 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.936590 29714 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.945662 29083 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.926903 29904 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:41.968356 29003 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.969949 29974 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.977998 29717 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:41.992631 28906 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.009963 29915 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:42.038378 30021 raft_consensus.cc:2243] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.039606 30021 raft_consensus.cc:2272] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.073509 30021 ts_tablet_manager.cc:1916] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:42.079602 29083 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:42.081290 29937 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:42.083244 29974 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.107002 29717 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:42.109884 30032 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:42.110690 29003 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.115609 29153 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:42.159265 29009 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.173241 29151 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:10:42.195871 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 81878.1us
I20260709 13:10:42.226186 29712 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:42.229619 30021 ts_tablet_manager.cc:1929] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:42.230101 30021 log.cc:1199] T eb070bde5f3744eb90ee5cf3dec8fcd4 P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/eb070bde5f3744eb90ee5cf3dec8fcd4
I20260709 13:10:42.232837 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet eb070bde5f3744eb90ee5cf3dec8fcd4 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:42.233744 29904 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:42.249831 30008 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.214810 28906 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:10:42.250810 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 85984.1us
I20260709 13:10:42.318470 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3aa255ba50b343eca7717515ec37d959 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:42.260923 29158 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.330677 29006 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:42.255739 29712 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.355934 30030 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:42.382827 30037 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.403759 29904 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:42.417171 29717 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.437014 29714 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.471015 28905 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:10:42.496388 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 110369us
I20260709 13:10:42.494637 28905 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:10:42.510852 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 85345.4us
I20260709 13:10:42.519767 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet 3aa255ba50b343eca7717515ec37d959 with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44418
I20260709 13:10:42.521752 30021 tablet_replica.cc:333] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:10:42.557283 29002 raft_consensus.cc:2955] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:10:42.568620 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 128633us
I20260709 13:10:42.630930 29151 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.645872 29078 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.673935 30021 raft_consensus.cc:2243] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.675011 30021 raft_consensus.cc:2272] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.679554 30021 ts_tablet_manager.cc:1916] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:42.684937 29003 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:42.693264 29714 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:42.695225 30037 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:42.696878 29079 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:42.699728 30021 ts_tablet_manager.cc:1929] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:42.699925 30049 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:42.700335 30021 log.cc:1199] T 3aa255ba50b343eca7717515ec37d959 P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/3aa255ba50b343eca7717515ec37d959
I20260709 13:10:42.709800 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet 3aa255ba50b343eca7717515ec37d959 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:42.714067 30032 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.726828 29079 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.733107 29717 raft_consensus.cc:2955] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.748314 28906 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.764791 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet 8da4f9ce63d54a63a8b9de11ba485699 with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35252
I20260709 13:10:42.771116 28892 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8da4f9ce63d54a63a8b9de11ba485699 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:42.823601 30052 tablet_replica.cc:333] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:10:42.833525 30052 raft_consensus.cc:2243] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.846575 30052 raft_consensus.cc:2272] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.876214 30052 ts_tablet_manager.cc:1916] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:42.897866 30052 ts_tablet_manager.cc:1929] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:42.898330 30052 log.cc:1199] T 8da4f9ce63d54a63a8b9de11ba485699 P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/8da4f9ce63d54a63a8b9de11ba485699
I20260709 13:10:42.899885 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet 8da4f9ce63d54a63a8b9de11ba485699 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:42.912115 29079 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:42.918388 29714 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:42.920034 29972 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:42.931113 29974 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.950959 29151 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:42.953568 29714 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.968487 30047 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:42.977147 29153 raft_consensus.cc:2955] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:42.986598 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6415e058581b4b7eb883c6e96b9fbfcf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:43.004288 28906 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:43.043629 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 6415e058581b4b7eb883c6e96b9fbfcf with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34914
I20260709 13:10:43.045696 30053 tablet_replica.cc:333] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:10:43.047022 30053 raft_consensus.cc:2243] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:43.047982 30053 raft_consensus.cc:2272] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:43.053048 30053 ts_tablet_manager.cc:1916] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:43.081219 30053 ts_tablet_manager.cc:1929] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:43.081701 30053 log.cc:1199] T 6415e058581b4b7eb883c6e96b9fbfcf P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/6415e058581b4b7eb883c6e96b9fbfcf
I20260709 13:10:43.083683 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 6415e058581b4b7eb883c6e96b9fbfcf (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:44.133111 29153 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:44.139797 29079 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:44.141510 29008 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:44.142891 30035 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:44.143905 29125 consensus_peers.cc:597] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b39693173a844463b5f0947904461289. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:44.143929 29975 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:44.158399 30035 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.159276 29083 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.171327 29008 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.177625 28906 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.196974 29083 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:44.239274 29151 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:10:44.239842 29051 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d989a55ed204649a0440142ce55e5ef. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:44.241480 30047 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:44.253849 29008 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:44.262434 30047 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:44.293370 30047 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.302480 29009 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.302384 29151 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.318548 28906 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.325160 29009 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:10:44.338569 28976 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 342b8b8f6e7742c897bcd3495fecb00a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:44.338765 29151 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:44.339805 29083 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:44.340889 30062 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:44.347710 30037 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:44.353526 30032 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.382576 29078 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:44.385015 29153 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.389814 29079 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.406119 28906 catalog_manager.cc:5697] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:44.433408 29051 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c141925650a74de58026be545d874618. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:44.434042 29158 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:44.434939 29008 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:44.435568 30066 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:44.442617 30047 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:44.469060 30066 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.472326 29009 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.477569 29153 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.504104 28906 catalog_manager.cc:5697] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:44.692498 30075 ts_tablet_manager.cc:933] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:44.696210 30075 tablet_copy_client.cc:323] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:10:44.698511 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet b39693173a844463b5f0947904461289 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:10:44.699072 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet b39693173a844463b5f0947904461289 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-b39693173a844463b5f0947904461289
I20260709 13:10:44.705612 29169 tablet_copy_source_session.cc:215] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:44.719338 30075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b39693173a844463b5f0947904461289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:44.741461 30075 tablet_copy_client.cc:806] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:44.742187 30075 tablet_copy_client.cc:670] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:44.770843 30075 tablet_copy_client.cc:538] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:44.789646 30075 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:44.826069 30080 ts_tablet_manager.cc:933] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:44.851838 30080 tablet_copy_client.cc:323] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:44.858349 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 8d989a55ed204649a0440142ce55e5ef from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:44.858907 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 8d989a55ed204649a0440142ce55e5ef from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-8d989a55ed204649a0440142ce55e5ef
I20260709 13:10:44.868535 29095 tablet_copy_source_session.cc:215] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:44.873507 30080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d989a55ed204649a0440142ce55e5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:44.878840 30079 ts_tablet_manager.cc:933] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:44.909289 30075 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:44.911290 30075 tablet_bootstrap.cc:492] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:44.908435 30079 tablet_copy_client.cc:323] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:44.915993 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet c141925650a74de58026be545d874618 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:44.916644 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet c141925650a74de58026be545d874618 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-c141925650a74de58026be545d874618
I20260709 13:10:44.927186 29095 tablet_copy_source_session.cc:215] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:44.915611 30075 ts_tablet_manager.cc:1403] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.126s	user 0.018s	sys 0.008s
I20260709 13:10:44.930725 30075 raft_consensus.cc:359] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:44.931349 30075 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:44.932117 30075 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:44.946266 30075 ts_tablet_manager.cc:1434] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.008s
I20260709 13:10:44.949172 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-b39693173a844463b5f0947904461289 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:10:44.949785 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-b39693173a844463b5f0947904461289 on tablet b39693173a844463b5f0947904461289 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:44.963348 30079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c141925650a74de58026be545d874618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:44.972532 30080 tablet_copy_client.cc:806] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:44.973196 30080 tablet_copy_client.cc:670] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:44.984160 30080 tablet_copy_client.cc:538] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:44.984789 30079 tablet_copy_client.cc:806] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:45.001052 30079 tablet_copy_client.cc:670] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:44.973798 30082 ts_tablet_manager.cc:933] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:10:45.016348 30079 tablet_copy_client.cc:538] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:45.066932 30082 tablet_copy_client.cc:323] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:10:45.071933 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet 342b8b8f6e7742c897bcd3495fecb00a from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:10:45.072506 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet 342b8b8f6e7742c897bcd3495fecb00a from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-342b8b8f6e7742c897bcd3495fecb00a
I20260709 13:10:45.078152 30079 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:45.077586 30080 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:45.083433 29020 tablet_copy_source_session.cc:215] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:45.086977 30082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342b8b8f6e7742c897bcd3495fecb00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.106343 30082 tablet_copy_client.cc:806] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:45.109304 30082 tablet_copy_client.cc:670] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
W20260709 13:10:45.113030 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 59581.3us
I20260709 13:10:45.116966 30082 tablet_copy_client.cc:538] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:10:45.127681 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 59159.5us
I20260709 13:10:45.133800 30082 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:45.152534 29714 raft_consensus.cc:1217] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:45.169595 30079 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:45.170585 30079 tablet_bootstrap.cc:492] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:45.171361 30079 ts_tablet_manager.cc:1403] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.093s	user 0.020s	sys 0.008s
I20260709 13:10:45.174424 30079 raft_consensus.cc:359] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:45.175458 30080 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:45.175292 30079 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:45.183816 30079 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:45.187243 30080 tablet_bootstrap.cc:492] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:45.188107 30080 ts_tablet_manager.cc:1403] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.111s	user 0.019s	sys 0.007s
I20260709 13:10:45.191498 30080 raft_consensus.cc:359] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:45.188419 30079 ts_tablet_manager.cc:1434] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.004s
I20260709 13:10:45.195336 30080 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:45.203248 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-c141925650a74de58026be545d874618 received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:45.204272 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-c141925650a74de58026be545d874618 on tablet c141925650a74de58026be545d874618 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:45.197517 30080 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:45.211594 30082 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:45.211741 30080 ts_tablet_manager.cc:1434] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 13:10:45.212617 30082 tablet_bootstrap.cc:492] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:45.213377 30082 ts_tablet_manager.cc:1403] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.080s	user 0.020s	sys 0.003s
I20260709 13:10:45.214048 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-8d989a55ed204649a0440142ce55e5ef received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:45.214655 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-8d989a55ed204649a0440142ce55e5ef on tablet 8d989a55ed204649a0440142ce55e5ef with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:45.216974 30082 raft_consensus.cc:359] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:45.217844 30082 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:45.218483 30082 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:45.221280 30082 ts_tablet_manager.cc:1434] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:10:45.224287 29020 tablet_copy_service.cc:342] P 06ae4197fd784525b695a9b16c8094ea: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-342b8b8f6e7742c897bcd3495fecb00a received from {username='slave'} at 127.0.0.1:33750
I20260709 13:10:45.224730 29020 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-342b8b8f6e7742c897bcd3495fecb00a on tablet 342b8b8f6e7742c897bcd3495fecb00a with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:45.231323 29714 raft_consensus.cc:1217] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:45.362943 29714 raft_consensus.cc:1217] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:45.416193 29714 raft_consensus.cc:1217] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:45.736919 30035 raft_consensus.cc:1064] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:45.740496 30035 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:45.767038 29009 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.768577 29975 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:45.773373 29078 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.773833 29714 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.779757 29975 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:45.781265 30093 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:45.844453 30094 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:45.851514 29078 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:45.859047 30066 raft_consensus.cc:1064] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:45.853935 29714 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:45.861634 30066 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:45.868726 29008 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:45.867105 28905 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:45.886082 29714 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.888365 30077 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:45.894515 29151 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.896335 30062 raft_consensus.cc:1064] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:45.896222 30077 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:45.899263 30062 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:45.912951 29003 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.915856 30077 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:45.919075 30066 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:45.930033 29078 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.931949 30102 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:45.933406 29717 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.932309 29151 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:45.935319 29714 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:45.947057 29153 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:45.939321 30087 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:45.972977 30108 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:45.979928 29003 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:45.991814 30099 raft_consensus.cc:1064] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:45.995201 30099 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:46.009315 30108 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.013041 29151 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.013677 29712 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.038563 29717 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:46.038856 29153 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:46.040444 30088 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:46.046146 29151 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:46.052031 29078 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.053356 30099 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:46.060715 29003 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:46.059249 28908 catalog_manager.cc:5697] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.062458 28908 catalog_manager.cc:5697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.064864 30107 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:46.090955 29003 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:46.105211 29151 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:46.110491 30087 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:46.113423 29714 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:46.113891 29712 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:46.116611 30093 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:46.119073 30087 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:46.118947 30099 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.126555 30108 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.129473 30035 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.135151 29714 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.138352 29713 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.144112 28892 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 342b8b8f6e7742c897bcd3495fecb00a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:46.154914 29152 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.160812 29712 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.162528 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b39693173a844463b5f0947904461289 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:46.164345 28905 catalog_manager.cc:5697] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:46.172303 29008 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.168396 29078 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:46.171096 29152 raft_consensus.cc:2955] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.184310 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet 342b8b8f6e7742c897bcd3495fecb00a with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44418
I20260709 13:10:46.194118 28907 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:46.199348 30094 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:46.204581 29078 raft_consensus.cc:2955] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.214471 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet b39693173a844463b5f0947904461289 with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34914
I20260709 13:10:46.216723 28905 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:46.233337 30120 tablet_replica.cc:333] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:10:46.235333 30120 raft_consensus.cc:2243] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.236210 30120 raft_consensus.cc:2272] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.234320 30121 tablet_replica.cc:333] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:10:46.237761 30121 raft_consensus.cc:2243] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.238457 30121 raft_consensus.cc:2272] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.248790 29078 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:46.253582 30120 ts_tablet_manager.cc:1916] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:46.254611 30121 ts_tablet_manager.cc:1916] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:46.256331 29712 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:46.257138 29002 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:46.257762 30107 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:46.258766 30088 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:46.266321 30066 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.267899 29712 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.275110 29003 raft_consensus.cc:2955] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.281596 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8d989a55ed204649a0440142ce55e5ef with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:46.288895 28907 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:46.309875 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d989a55ed204649a0440142ce55e5ef with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35252
I20260709 13:10:46.314353 30120 ts_tablet_manager.cc:1929] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:46.315860 30120 log.cc:1199] T 342b8b8f6e7742c897bcd3495fecb00a P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/342b8b8f6e7742c897bcd3495fecb00a
I20260709 13:10:46.315418 30121 ts_tablet_manager.cc:1929] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:46.317471 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet 342b8b8f6e7742c897bcd3495fecb00a (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:46.317862 30121 log.cc:1199] T b39693173a844463b5f0947904461289 P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/b39693173a844463b5f0947904461289
I20260709 13:10:46.324751 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet b39693173a844463b5f0947904461289 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:46.334908 30122 tablet_replica.cc:333] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:10:46.336109 30122 raft_consensus.cc:2243] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.336840 30122 raft_consensus.cc:2272] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.340432 30122 ts_tablet_manager.cc:1916] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:46.360159 30122 ts_tablet_manager.cc:1929] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:46.360666 30122 log.cc:1199] T 8d989a55ed204649a0440142ce55e5ef P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/8d989a55ed204649a0440142ce55e5ef
I20260709 13:10:46.362548 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet 8d989a55ed204649a0440142ce55e5ef (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:46.484148 29078 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:46.493248 29712 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:46.493300 29002 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:46.495108 30077 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:46.495942 30107 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:46.504920 30088 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.505586 29712 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.509459 29002 raft_consensus.cc:2955] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.527139 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c141925650a74de58026be545d874618 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:46.528419 28906 catalog_manager.cc:5697] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:46.538944 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet c141925650a74de58026be545d874618 with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35252
I20260709 13:10:46.539930 30122 tablet_replica.cc:333] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:10:46.540845 30122 raft_consensus.cc:2243] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:46.541461 30122 raft_consensus.cc:2272] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:46.544553 30122 ts_tablet_manager.cc:1916] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:46.558175 30122 ts_tablet_manager.cc:1929] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:46.558641 30122 log.cc:1199] T c141925650a74de58026be545d874618 P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/c141925650a74de58026be545d874618
I20260709 13:10:46.560200 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet c141925650a74de58026be545d874618 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:47.613145 29152 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:47.618744 29002 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:47.618825 29078 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:10:47.619024 29125 consensus_peers.cc:597] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d4c87bb2e4a4f2ca684feff1fae17cf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:47.620091 29975 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.620766 30123 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.627792 29975 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.630309 29002 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.641911 29079 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:47.635907 29078 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.650712 29008 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:10:47.649860 29051 consensus_peers.cc:597] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8f1b4bfdf0b2488586aa6cbe1ca64200. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:47.655047 30107 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:47.659392 28908 catalog_manager.cc:5697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.661130 29153 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:47.664542 30066 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.678337 30107 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.685230 29008 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.692605 29152 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.715886 28906 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.718516 29078 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:10:47.726984 29051 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00715172c7364d849f262c553c53e5fe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:47.727339 29008 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:47.728372 29153 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:47.728530 30107 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.729818 30066 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.740715 30107 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.743263 29153 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.747036 29008 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.756520 28907 catalog_manager.cc:5697] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.760871 29153 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:10:47.773290 29125 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f97f68627cfa4585b418a3c090e44c14. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:47.774734 29078 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:47.775964 30123 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.778425 29008 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:47.779837 30123 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.790756 29975 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.793565 29078 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.796921 29002 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:47.810513 28906 catalog_manager.cc:5697] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.034511 30141 ts_tablet_manager.cc:933] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:48.044381 30141 tablet_copy_client.cc:323] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:10:48.046373 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:10:48.046840 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet 3d4c87bb2e4a4f2ca684feff1fae17cf from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-3d4c87bb2e4a4f2ca684feff1fae17cf
I20260709 13:10:48.052583 29169 tablet_copy_source_session.cc:215] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:48.056133 30141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:48.071328 30141 tablet_copy_client.cc:806] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:48.072033 30141 tablet_copy_client.cc:670] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:48.076105 30141 tablet_copy_client.cc:538] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:48.083312 30141 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:48.101058 30141 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:48.101819 30141 tablet_bootstrap.cc:492] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:48.102357 30141 ts_tablet_manager.cc:1403] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.019s	user 0.019s	sys 0.000s
I20260709 13:10:48.104574 30141 raft_consensus.cc:359] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:48.105173 30141 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:48.105813 30141 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:48.108980 30141 ts_tablet_manager.cc:1434] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.005s
I20260709 13:10:48.114583 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-3d4c87bb2e4a4f2ca684feff1fae17cf received from {username='slave'} at 127.0.0.1:32836
I20260709 13:10:48.115033 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-3d4c87bb2e4a4f2ca684feff1fae17cf on tablet 3d4c87bb2e4a4f2ca684feff1fae17cf with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:48.149847 30141 ts_tablet_manager.cc:933] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:48.151485 30141 tablet_copy_client.cc:323] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:48.152949 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 00715172c7364d849f262c553c53e5fe from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:48.153469 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 00715172c7364d849f262c553c53e5fe from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-00715172c7364d849f262c553c53e5fe
I20260709 13:10:48.158787 29095 tablet_copy_source_session.cc:215] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:48.161733 30141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00715172c7364d849f262c553c53e5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:48.178625 30141 tablet_copy_client.cc:806] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:48.179302 30141 tablet_copy_client.cc:670] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:48.184614 30141 tablet_copy_client.cc:538] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:48.195652 30141 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:48.232755 30141 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:48.233706 30141 tablet_bootstrap.cc:492] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:48.234390 30141 ts_tablet_manager.cc:1403] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.039s	user 0.026s	sys 0.007s
I20260709 13:10:48.239086 30148 ts_tablet_manager.cc:933] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:48.237919 30141 raft_consensus.cc:359] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:48.239826 30141 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:48.240477 30141 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:48.242292 30148 tablet_copy_client.cc:323] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:48.242985 30141 ts_tablet_manager.cc:1434] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:10:48.245982 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-00715172c7364d849f262c553c53e5fe received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:48.246433 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-00715172c7364d849f262c553c53e5fe on tablet 00715172c7364d849f262c553c53e5fe with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:48.252254 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:48.252696 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-8f1b4bfdf0b2488586aa6cbe1ca64200
I20260709 13:10:48.258867 29095 tablet_copy_source_session.cc:215] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:48.263482 30148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:48.280486 30148 tablet_copy_client.cc:806] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:48.281054 30148 tablet_copy_client.cc:670] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:48.284488 30148 tablet_copy_client.cc:538] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:48.293335 30148 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:48.318387 30148 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:48.320760 30148 tablet_bootstrap.cc:492] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:48.321625 30148 ts_tablet_manager.cc:1403] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.004s
I20260709 13:10:48.325657 30148 raft_consensus.cc:359] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:48.326794 30148 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:48.327464 30148 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:48.331939 30148 ts_tablet_manager.cc:1434] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.006s
I20260709 13:10:48.333767 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-8f1b4bfdf0b2488586aa6cbe1ca64200 received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:48.334192 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-8f1b4bfdf0b2488586aa6cbe1ca64200 on tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:48.348241 30148 ts_tablet_manager.cc:933] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:48.349794 30148 tablet_copy_client.cc:323] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:10:48.351305 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet f97f68627cfa4585b418a3c090e44c14 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:10:48.351833 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet f97f68627cfa4585b418a3c090e44c14 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-f97f68627cfa4585b418a3c090e44c14
I20260709 13:10:48.359556 29169 tablet_copy_source_session.cc:215] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:48.362610 30148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97f68627cfa4585b418a3c090e44c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:48.380081 30148 tablet_copy_client.cc:806] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:48.380689 30148 tablet_copy_client.cc:670] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:48.385587 30148 tablet_copy_client.cc:538] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:48.393915 30148 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:48.418224 30148 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:48.420186 30148 tablet_bootstrap.cc:492] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:48.420903 30148 ts_tablet_manager.cc:1403] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.005s
I20260709 13:10:48.423991 30148 raft_consensus.cc:359] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:48.424692 30148 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:48.425222 30148 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:48.429483 30148 ts_tablet_manager.cc:1434] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:10:48.431166 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-f97f68627cfa4585b418a3c090e44c14 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:10:48.431604 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-f97f68627cfa4585b418a3c090e44c14 on tablet f97f68627cfa4585b418a3c090e44c14 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:48.586982 29712 raft_consensus.cc:1217] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:48.629177 29712 raft_consensus.cc:1217] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:48.665234 29712 raft_consensus.cc:1217] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:48.729408 29712 raft_consensus.cc:1217] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:49.092597 30128 raft_consensus.cc:1064] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:49.094761 30128 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:49.107146 29714 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.107705 29152 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.109850 29002 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.110369 30128 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.111603 30066 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.113080 30158 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.140316 30066 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.142839 29152 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.145762 30158 raft_consensus.cc:1064] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:49.150317 30158 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:49.154263 29714 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.159751 28908 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:49.179136 30094 raft_consensus.cc:1064] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:49.181730 30094 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:49.187901 29002 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.212877 29714 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.214521 30123 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.216823 29002 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.217698 29008 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.218241 30128 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.218639 29079 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.219111 30154 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.219539 29152 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.221400 29714 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.224854 30153 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.226178 30066 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.233404 29079 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:49.244724 30123 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:49.251008 29713 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:49.253118 30128 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:49.253688 29153 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:49.260285 30128 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:49.273198 30159 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.278841 29153 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.284250 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:49.287675 29713 raft_consensus.cc:2955] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.299168 28905 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:49.305526 30153 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.311569 29712 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.312847 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34914
I20260709 13:10:49.305413 29152 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.319377 29008 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.325302 30176 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.337179 28905 catalog_manager.cc:5697] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:49.348320 30123 raft_consensus.cc:1064] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:49.350728 29714 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.354763 30123 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:49.366178 29078 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.371929 29079 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:49.386444 30178 tablet_replica.cc:333] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:10:49.390856 30178 raft_consensus.cc:2243] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.393958 30178 raft_consensus.cc:2272] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.386497 29009 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:49.408689 30066 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:49.401602 28907 catalog_manager.cc:5697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.414372 30178 ts_tablet_manager.cc:1916] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:49.422171 30159 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.448725 29003 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.425236 29002 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.450433 29712 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.455014 29151 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:49.456261 29078 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:49.452577 30169 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:10:49.435333 29009 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.437655 29717 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:49.462880 30158 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:49.468920 29717 raft_consensus.cc:2955] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.469856 29712 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:49.471958 29078 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:49.475735 30175 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.480764 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 00715172c7364d849f262c553c53e5fe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:49.487476 30123 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:49.489897 30169 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:49.508284 30169 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:49.513041 28907 catalog_manager.cc:5697] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 3 to 4, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.513465 30176 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:10:49.563644 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54133us
I20260709 13:10:49.573500 30178 ts_tablet_manager.cc:1929] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:49.574059 30178 log.cc:1199] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/8f1b4bfdf0b2488586aa6cbe1ca64200
I20260709 13:10:49.575816 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 8f1b4bfdf0b2488586aa6cbe1ca64200 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:49.587334 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet 00715172c7364d849f262c553c53e5fe with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35252
I20260709 13:10:49.587957 30154 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.588609 29079 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.604488 30184 tablet_replica.cc:333] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:10:49.605779 30184 raft_consensus.cc:2243] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.606570 30184 raft_consensus.cc:2272] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.604846 29009 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.611593 30184 ts_tablet_manager.cc:1916] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:49.621006 28907 catalog_manager.cc:5697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.622311 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:49.626798 30184 ts_tablet_manager.cc:1929] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:49.627313 30184 log.cc:1199] T 00715172c7364d849f262c553c53e5fe P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/00715172c7364d849f262c553c53e5fe
I20260709 13:10:49.628386 28905 catalog_manager.cc:5697] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.632354 29080 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.637434 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d4c87bb2e4a4f2ca684feff1fae17cf with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34914
I20260709 13:10:49.639006 30178 tablet_replica.cc:333] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:10:49.640430 30178 raft_consensus.cc:2243] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.641148 30178 raft_consensus.cc:2272] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.643451 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet 00715172c7364d849f262c553c53e5fe (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:49.645195 29712 raft_consensus.cc:2955] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.652729 29713 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.675730 30178 ts_tablet_manager.cc:1916] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:49.689890 29152 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:49.701494 29713 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:49.703163 30094 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:49.705027 29009 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:49.706508 30178 ts_tablet_manager.cc:1929] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:49.706954 30094 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:49.707057 30178 log.cc:1199] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/3d4c87bb2e4a4f2ca684feff1fae17cf
I20260709 13:10:49.708720 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 3d4c87bb2e4a4f2ca684feff1fae17cf (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:49.713030 30154 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.716285 29712 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.718513 29002 raft_consensus.cc:2955] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.732396 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f97f68627cfa4585b418a3c090e44c14 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:49.733560 28908 catalog_manager.cc:5697] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 3 to 4, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:49.745026 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet f97f68627cfa4585b418a3c090e44c14 with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44418
I20260709 13:10:49.747287 30185 tablet_replica.cc:333] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:10:49.748457 30185 raft_consensus.cc:2243] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:49.749225 30185 raft_consensus.cc:2272] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:49.754765 30185 ts_tablet_manager.cc:1916] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:49.766506 30185 ts_tablet_manager.cc:1929] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:49.766963 30185 log.cc:1199] T f97f68627cfa4585b418a3c090e44c14 P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/f97f68627cfa4585b418a3c090e44c14
I20260709 13:10:49.768710 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet f97f68627cfa4585b418a3c090e44c14 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:10:49.772424 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:49.824234 29080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:49.824949 29080 raft_consensus.cc:606] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:49.840669 29152 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:10:49.841172 29152 raft_consensus.cc:606] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Received request to transfer leadership to TS ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:49.843943 29152 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a08e186ea0749ae979329103591a933"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:10:49.844530 29152 raft_consensus.cc:606] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:49.848565 29080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:49.849202 29080 raft_consensus.cc:606] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Received request to transfer leadership to TS ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:49.852737 29152 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:10:49.853286 29152 raft_consensus.cc:606] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Received request to transfer leadership to TS ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:49.854698 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:49.856122 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:49.893601 30154 raft_consensus.cc:993] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:49.894014 30154 raft_consensus.cc:1081] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:49.895398 29009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a08e186ea0749ae979329103591a933"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:10:49.895902 29009 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:49.896215 29009 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:49.901772 29009 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:49.903961 29009 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:49.904632 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:49.905025 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:49.905543 29080 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:49.905602 29158 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:49.906096 29158 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:49.906929 29158 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:49.908023 29158 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:49.912530 29080 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:49.913219 29158 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:49.913671 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:49.914425 30155 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:49.914880 30155 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:49.915679 30155 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:49.925333 28905 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 2, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 2 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.012540 30159 raft_consensus.cc:993] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:50.013087 30181 raft_consensus.cc:1081] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:50.015056 29009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34922
I20260709 13:10:50.015652 29009 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:50.016081 29009 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.021847 29009 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:50.023502 30176 raft_consensus.cc:993] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:50.023713 29009 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:50.024143 30176 raft_consensus.cc:1081] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Signalling peer ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:50.024511 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:50.025126 29080 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:50.025483 29080 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:50.025524 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:50.025880 29713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:49414
I20260709 13:10:50.026198 29158 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.026602 29713 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:50.027016 29713 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.026468 29080 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:50.031997 29158 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:50.032763 29080 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.033294 28975 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:50.034134 30155 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:50.034668 30155 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:50.035574 30155 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:50.038600 29080 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:50.041358 29713 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.045287 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:50.045665 29713 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:50.045855 29158 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:50.046236 29158 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:50.044772 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:50.046940 29080 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.046940 29158 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.048101 29158 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.051569 28905 catalog_manager.cc:5697] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 2 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.054349 29080 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:50.055371 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:50.056093 30144 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:50.057255 30144 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:50.058177 30144 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.061668 29158 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:50.068965 28905 catalog_manager.cc:5697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.102082 30181 raft_consensus.cc:993] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: : Instructing follower ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:50.102741 30153 raft_consensus.cc:1081] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Signalling peer ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:50.104255 29712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:47384
I20260709 13:10:50.104700 29712 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:50.104964 29712 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.109696 29712 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.111155 29712 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:50.111924 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:50.112134 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:50.112552 29080 raft_consensus.cc:3055] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:50.112614 29158 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.112931 29080 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:50.113687 29080 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.114790 29080 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.121148 29158 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:50.122786 29682 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:50.123600 30144 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:50.124064 30144 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:50.125119 30144 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.125425 29080 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:50.132506 28905 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.295786 29080 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:50.297236 30155 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.327216 30176 raft_consensus.cc:993] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:50.327831 30176 raft_consensus.cc:1081] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Signalling peer ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:50.328367 29158 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:50.330112 29713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:49414
I20260709 13:10:50.330658 29713 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:50.331036 29713 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.330868 30197 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.338552 29713 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.342289 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:50.342787 29008 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.343428 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:50.343981 29158 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:50.344316 29158 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:50.345187 29158 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.346721 29158 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.349252 29008 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:50.342734 29713 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:50.353838 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:50.354839 30144 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:50.355324 30144 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:50.356196 30144 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:50.372650 29158 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:50.379297 28905 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.449721 29079 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:50.451056 30144 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:50.460498 29158 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:50.465241 30191 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:50.474145 29152 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:50.475704 30197 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:50.504266 29079 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:50.505923 30196 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.567258 29152 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:50.568490 30144 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:50.576915 29079 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:50.585878 30191 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:50.798722 29152 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:50.799880 30206 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:50.815472 29008 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:50.820535 30206 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:50.828022 29079 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:10:50.837898 29051 consensus_peers.cc:597] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ed915f0be134d71b36d8fc8f7b5830b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.838876 29008 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:50.839059 29152 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:50.840107 30202 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.840900 30153 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.848445 30159 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:50.849835 29008 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:50.853307 29152 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:50.857100 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:50.861061 29158 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:50.871475 28907 catalog_manager.cc:5697] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:50.892272 29125 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab75774d33164f0b9c0269be22426768. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.897367 29008 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:50.899808 30154 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.904698 29079 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:50.910043 30154 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.917771 30176 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:50.921625 29008 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:50.924645 29079 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:50.946089 29008 consensus_queue.cc:237] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:50.946933 28908 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:50.956342 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:50.963903 29158 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:10:50.964920 28976 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2185927c34434da38aa9e7776e483b1b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.967259 29080 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:50.969188 30196 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.970615 30197 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.998591 30196 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:51.010150 29153 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:51.028858 29080 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.030370 29079 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:51.034403 28908 catalog_manager.cc:5697] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:51.051743 29051 consensus_peers.cc:597] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 027a3f0877db4e259d94a8e296822644. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:51.052295 29153 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:51.055168 29008 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:51.060171 30202 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:51.063189 30159 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:51.064419 29079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "243d4ae4a4ae48b2979e32874472d256"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:51.065091 29079 raft_consensus.cc:606] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Received request to transfer leadership to TS 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:51.083941 29080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:51.085309 29080 raft_consensus.cc:606] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:51.091523 29080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:51.092744 29080 raft_consensus.cc:606] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Received request to transfer leadership to TS ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:51.098289 29079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:51.098930 29079 raft_consensus.cc:606] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:51.102247 29080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c141925650a74de58026be545d874618"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:51.102864 29080 raft_consensus.cc:606] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Received request to transfer leadership to TS ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:51.104512 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:51.103302 30220 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:51.106226 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:51.105407 29153 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:51.109369 29009 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:10:51.124504 28906 catalog_manager.cc:5697] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 2, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 1 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.163723 30153 raft_consensus.cc:993] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: : Instructing follower ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:51.164233 30153 raft_consensus.cc:1081] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Signalling peer ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:51.167243 29712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c141925650a74de58026be545d874618"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:47384
I20260709 13:10:51.167773 29712 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.168130 29712 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.176079 29712 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:51.178030 29712 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:51.178725 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:51.179054 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:51.179328 29009 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.179572 29079 raft_consensus.cc:3055] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:51.179874 29079 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:51.180619 29079 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:51.181802 29079 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.188936 29079 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:51.190166 29683 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:51.191082 30206 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:51.191589 30206 raft_consensus.cc:697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:51.192497 30206 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:51.193506 29009 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:51.203061 28906 catalog_manager.cc:5697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:51.220696 30224 ts_tablet_manager.cc:933] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:51.223073 30224 tablet_copy_client.cc:323] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:51.224718 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 2ed915f0be134d71b36d8fc8f7b5830b from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:51.225243 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 2ed915f0be134d71b36d8fc8f7b5830b from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-2ed915f0be134d71b36d8fc8f7b5830b
I20260709 13:10:51.232992 29095 tablet_copy_source_session.cc:215] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:51.236370 30224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ed915f0be134d71b36d8fc8f7b5830b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.254328 30224 tablet_copy_client.cc:806] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:51.255043 30224 tablet_copy_client.cc:670] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:51.258728 30224 tablet_copy_client.cc:538] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:51.267017 30224 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:51.284451 30224 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:51.285401 30224 tablet_bootstrap.cc:492] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:51.286103 30224 ts_tablet_manager.cc:1403] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.008s
I20260709 13:10:51.289021 30224 raft_consensus.cc:359] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.289867 30224 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:51.290537 30224 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.292992 30224 ts_tablet_manager.cc:1434] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:51.306082 30227 ts_tablet_manager.cc:933] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:51.306571 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-2ed915f0be134d71b36d8fc8f7b5830b received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:51.307090 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-2ed915f0be134d71b36d8fc8f7b5830b on tablet 2ed915f0be134d71b36d8fc8f7b5830b with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:51.310886 30227 tablet_copy_client.cc:323] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:10:51.312733 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet ab75774d33164f0b9c0269be22426768 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:10:51.313387 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet ab75774d33164f0b9c0269be22426768 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-ab75774d33164f0b9c0269be22426768
I20260709 13:10:51.318969 29169 tablet_copy_source_session.cc:215] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:51.321749 30227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab75774d33164f0b9c0269be22426768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.335253 30227 tablet_copy_client.cc:806] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:51.335794 30227 tablet_copy_client.cc:670] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:51.339773 30227 tablet_copy_client.cc:538] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:51.341022 30153 raft_consensus.cc:993] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:51.341771 30153 raft_consensus.cc:1081] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:51.346869 30153 raft_consensus.cc:993] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: : Instructing follower ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:51.347414 30220 raft_consensus.cc:1081] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Signalling peer ca45420e033847489ad989c1c6eed3ef to start an election
I20260709 13:10:51.348937 30227 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:51.349246 29712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:47384
I20260709 13:10:51.349793 29712 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.350122 29712 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.351233 29009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34922
I20260709 13:10:51.351784 29009 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.354461 30153 raft_consensus.cc:993] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: : Instructing follower 243d4ae4a4ae48b2979e32874472d256 to start an election
I20260709 13:10:51.354661 29009 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.355036 30159 raft_consensus.cc:1081] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Signalling peer 243d4ae4a4ae48b2979e32874472d256 to start an election
I20260709 13:10:51.359674 29712 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:51.360245 29153 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
 from {username='slave'} at 127.0.0.1:35260
I20260709 13:10:51.360798 29153 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.361166 29153 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.362564 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:51.362725 29712 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:51.363178 29079 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:51.363615 29079 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:51.364605 29079 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:51.367836 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:51.372575 29152 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.374365 29153 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.377738 29153 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:51.378269 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:51.379061 29008 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.385747 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:51.386476 29080 raft_consensus.cc:3055] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:51.386785 29080 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:51.387483 29080 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.389448 29080 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.366211 29079 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.416229 29009 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:51.419589 29152 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:51.422504 29080 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:51.430846 29009 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:51.428869 29079 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:51.435194 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:51.436281 30144 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:51.436786 30144 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:51.437666 30144 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:51.440415 30227 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:51.441591 30227 tablet_bootstrap.cc:492] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:51.442329 30227 ts_tablet_manager.cc:1403] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.094s	user 0.045s	sys 0.045s
I20260709 13:10:51.444638 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:51.445232 29153 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.445644 30227 raft_consensus.cc:359] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.446619 30227 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:51.447311 30227 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.449848 30227 ts_tablet_manager.cc:1434] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:10:51.451468 29153 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:51.452777 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:51.453598 30087 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:51.454078 30087 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:51.454900 30087 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:51.462226 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 58605.4us
I20260709 13:10:51.465698 30224 ts_tablet_manager.cc:933] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:10:51.472657 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:51.475878 29008 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:51.477331 30195 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:51.488969 30195 raft_consensus.cc:697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:51.489897 30195 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.478852 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-ab75774d33164f0b9c0269be22426768 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:10:51.491300 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-ab75774d33164f0b9c0269be22426768 on tablet ab75774d33164f0b9c0269be22426768 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:51.481348 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:51.492363 29079 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:51.492666 29079 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:51.493295 29079 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:51.495324 29079 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.501605 29079 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:51.509894 28906 catalog_manager.cc:5697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.522707 28905 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:51.534044 30224 tablet_copy_client.cc:323] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:10:51.539763 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet 2185927c34434da38aa9e7776e483b1b from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:10:51.540369 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet 2185927c34434da38aa9e7776e483b1b from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-2185927c34434da38aa9e7776e483b1b
I20260709 13:10:51.540493 30220 raft_consensus.cc:993] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:51.540988 30220 raft_consensus.cc:1081] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:51.545243 29003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34922
I20260709 13:10:51.545742 29003 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.546108 29003 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.554653 29003 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:51.560667 29020 tablet_copy_source_session.cc:215] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:51.566970 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:51.567425 29003 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:51.570992 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:51.571568 29079 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:51.571853 29079 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:51.572553 29079 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:51.573593 29079 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.570503 30224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2185927c34434da38aa9e7776e483b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.567595 29153 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.583925 29079 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:51.589816 29153 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:51.590940 28975 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:51.591787 30196 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:51.592334 30196 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:51.593122 30196 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:51.597026 30227 ts_tablet_manager.cc:933] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:51.601398 28905 catalog_manager.cc:5697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 2 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.591849 30224 tablet_copy_client.cc:806] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:51.605341 30224 tablet_copy_client.cc:670] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:51.613510 30224 tablet_copy_client.cc:538] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:51.623234 30224 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:51.622267 28905 catalog_manager.cc:5697] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 2 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.623975 30227 tablet_copy_client.cc:323] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:10:51.625636 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 027a3f0877db4e259d94a8e296822644 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:10:51.626176 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 027a3f0877db4e259d94a8e296822644 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-027a3f0877db4e259d94a8e296822644
I20260709 13:10:51.630973 29003 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:51.633008 30144 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:51.641810 29095 tablet_copy_source_session.cc:215] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:51.646299 30227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027a3f0877db4e259d94a8e296822644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.653586 29079 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:51.655236 30191 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:51.657256 30224 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:51.658265 30224 tablet_bootstrap.cc:492] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:51.658967 30224 ts_tablet_manager.cc:1403] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.014s
I20260709 13:10:51.662789 30224 raft_consensus.cc:359] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.664871 30224 raft_consensus.cc:740] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:51.665663 30224 consensus_queue.cc:260] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.668442 30224 ts_tablet_manager.cc:1434] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.002s
I20260709 13:10:51.676713 29020 tablet_copy_service.cc:342] P 06ae4197fd784525b695a9b16c8094ea: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-2185927c34434da38aa9e7776e483b1b received from {username='slave'} at 127.0.0.1:33750
I20260709 13:10:51.677210 29020 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-2185927c34434da38aa9e7776e483b1b on tablet 2185927c34434da38aa9e7776e483b1b with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:51.705226 30227 tablet_copy_client.cc:806] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:51.706040 30227 tablet_copy_client.cc:670] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:51.713363 30227 tablet_copy_client.cc:538] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:51.723079 30227 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:10:51.741472 30227 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:51.742379 30227 tablet_bootstrap.cc:492] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:10:51.743073 30227 ts_tablet_manager.cc:1403] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.005s
I20260709 13:10:51.746062 30227 raft_consensus.cc:359] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.746748 30227 raft_consensus.cc:740] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:10:51.747300 30227 consensus_queue.cc:260] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:51.750738 30227 ts_tablet_manager.cc:1434] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:51.753279 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-027a3f0877db4e259d94a8e296822644 received from {username='slave'} at 127.0.0.1:39900
I20260709 13:10:51.753718 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-027a3f0877db4e259d94a8e296822644 on tablet 027a3f0877db4e259d94a8e296822644 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:51.895179 29003 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:10:51.896560 30235 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:51.925104 29712 raft_consensus.cc:1217] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:51.932382 29712 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Advancing to term 2
I20260709 13:10:51.932706 29713 raft_consensus.cc:1217] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:51.948622 29078 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:10:51.953051 29712 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:10:51.960103 29153 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:51.953143 30176 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:10:51.961685 30191 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:51.962054 30154 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:52.017191 29153 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:52.017572 29078 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:52.019133 30196 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:52.028121 29079 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:52.032584 30234 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:10:52.037846 30191 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:10:52.067113 29083 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:52.079330 30176 raft_consensus.cc:1064] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:52.079402 30196 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:52.095512 30176 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:52.209000 29717 raft_consensus.cc:1217] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:52.210685 29712 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:10:52.216224 30195 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:52.465754 29008 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:10:52.467087 29078 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:10:52.468317 30176 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:53.075649 30234 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.078516 30234 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.118554 30256 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.119729 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:10:53.135393 29083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.052s	user 0.008s	sys 0.012s
W20260709 13:10:53.135970 29083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.009s	sys 0.012s
I20260709 13:10:53.136312 30234 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.136972 30234 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.137559 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 776) took 1055 ms. Trace:
I20260709 13:10:53.137727 30234 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.150513 30234 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.156322 30234 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:53.133145 30256 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.162681 30256 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.183733 30256 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.138255 29083 rpcz_store.cc:276] 0709 13:10:52.082087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:52.082222 (+   135us) service_pool.cc:224] Handling call
0709 13:10:52.082525 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:52.082839 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:52.082869 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:52.083241 (+   372us) log.cc:844] Serialized 26 byte log entry
0709 13:10:53.136793 (+1053552us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:10:53.136881 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:53.137033 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:53.137138 (+   105us) raft_consensus.cc:1685] finished
0709 13:10:53.137155 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:53.137400 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":425,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":94,"thread_start_us":1052868,"threads_started":1,"wal-append.queue_time_us":132241}
I20260709 13:10:53.198640 30258 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.199640 30258 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.201471 30259 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:10:53.201750 29153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.129s	user 0.047s	sys 0.069s
I20260709 13:10:53.202257 30259 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.207183 30258 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.211416 30263 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.212399 30263 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.215555 30263 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:53.207875 29153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.135s	user 0.049s	sys 0.070s
I20260709 13:10:53.224306 30257 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.225117 30257 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.185853 30256 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.233738 30159 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.234934 30159 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.237097 30258 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.237900 30258 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.238412 30259 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.239056 30259 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.241051 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 851) took 1214 ms. Trace:
I20260709 13:10:53.241496 29153 rpcz_store.cc:276] 0709 13:10:52.026578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:52.026684 (+   106us) service_pool.cc:224] Handling call
0709 13:10:52.037150 (+ 10466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:52.038473 (+  1323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:52.038501 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:52.072710 (+ 34209us) log.cc:844] Serialized 26 byte log entry
0709 13:10:53.240432 (+1167722us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:10:53.240500 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:53.240613 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:53.240651 (+    38us) raft_consensus.cc:1685] finished
0709 13:10:53.240662 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:53.240825 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1094,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":367,"thread_start_us":1148491,"threads_started":1,"wal-append.queue_time_us":371661}
I20260709 13:10:53.242717 30258 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.246978 30263 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.247843 30263 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.251714 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:53.251708 30258 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.255721 30258 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.256747 30257 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.283238 30233 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.308248 30087 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.345422 30268 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.346261 30268 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.350209 30268 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.268241 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:53.357439 30272 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.358517 30272 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.367424 30266 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.337464 30258 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.350730 30263 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.239876 30259 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.367841 30159 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.362458 30274 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.367188 30233 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.327930 30265 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.375826 30274 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.378444 30276 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.370256 30255 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.377923 30087 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.383152 30261 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.364959 30257 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.383883 30261 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.379124 30159 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.370267 30267 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.399834 30087 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.390848 30260 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.396287 30256 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.397676 30276 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.403885 30159 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.391804 30258 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.406524 30257 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.408416 30267 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.412987 30255 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.428169 30270 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.428884 30270 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.429566 30255 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.400568 30265 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.434095 30265 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.400435 30266 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.437206 30266 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.440161 30271 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.441077 30271 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.444885 30282 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.446605 30282 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.445811 30280 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.449368 30280 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.453110 30285 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.454216 30285 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:53.414036 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 96645.5us
I20260709 13:10:53.461458 30263 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.462339 30263 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.463325 30235 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.464818 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:53.465056 30274 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.424516 30273 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.486271 30272 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.460273 30255 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.488692 30255 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.489120 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:53.439174 30260 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.484246 30087 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.502720 30159 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.458379 30258 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.490093 30266 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.490372 30279 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.591379 30087 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.652144 30287 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.505973 30235 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.676327 30263 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.683692 30286 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.661654 30233 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.604828 30266 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.698915 30280 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.706272 30272 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.700714 30159 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.674049 30196 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.693547 30263 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.590148 30273 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.658802 30287 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.696815 30257 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.706919 30272 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.711897 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:53.719264 30159 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.720822 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:53.676620 30293 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.710614 30280 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.710214 30233 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.722174 30285 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.712718 30265 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.723302 29717 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:53.710004 30282 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.722994 29712 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:53.684535 30286 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.719798 30274 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.718106 30261 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.764302 30284 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.761569 30233 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.766912 30295 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.740787 30255 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.744488 30281 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:53.733227 30235 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.771667 30159 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.722457 29715 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:53.748726 30271 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.741098 30263 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.779277 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:53.741637 30257 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.672998 30279 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.770711 30276 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.786906 30283 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.789040 30292 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.776826 30295 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.794802 30290 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.795076 30260 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.761754 30196 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.786144 30159 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.795677 30290 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.783746 30282 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.809160 30256 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.811172 30256 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.812448 30290 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.813257 30290 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.813973 30256 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.815457 30159 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.801362 30297 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.790731 30258 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.784569 30288 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.801373 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:53.801012 30196 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.789117 30283 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:53.790493 30274 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.866132 30159 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.869019 30288 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.872430 30288 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.798915 30277 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.874608 30159 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:53.796779 30265 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.800308 30289 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.800689 30257 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.857755 30258 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.801813 30280 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:53.884596 30159 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.866688 30273 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.895059 30258 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.895751 30287 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.898929 30265 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.880554 30255 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.902489 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:53.790022 30292 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.902024 30283 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.881422 30286 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.902913 30258 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.814028 30290 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.913370 29079 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:53.802835 30281 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.816716 30297 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:53.900702 30266 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.801667 30284 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:53.787530 30293 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.919442 30159 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.915786 30255 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.891196 30282 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.922652 30279 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.927171 30292 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.927914 30289 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.918066 30272 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.157418 30234 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.927219 30255 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.939944 30234 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.940495 30284 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.920547 30258 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.941321 30258 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.943521 30258 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.944492 30233 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.944658 30257 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.916604 30277 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.948423 30282 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.949980 30277 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.955958 30283 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:53.946250 30272 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.957271 30283 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:53.958556 30267 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.961854 30297 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.963248 30295 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.964370 30280 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.965092 30272 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:53.967681 30272 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:53.969791 30293 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:53.950028 30281 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.970783 30259 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.959762 30283 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.995548 30300 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:53.997311 30259 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:53.998026 30259 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.000825 30283 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.001554 30283 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.003774 30283 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:53.996335 30300 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.008917 30266 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.009526 30266 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.011502 30266 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.005921 30259 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.015250 30301 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.013185 30287 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.019307 30287 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.021345 30287 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.015892 30301 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.025022 30301 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.026175 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 30) took 1575 ms. Trace:
I20260709 13:10:54.027455 29003 rpcz_store.cc:276] 0709 13:10:52.450697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:53.198174 (+747477us) service_pool.cc:224] Handling call
0709 13:10:53.198675 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:53.944143 (+745468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:53.944174 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:53.944231 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:53.944260 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:54.025745 (+ 81485us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400026054. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:10:54.025801 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:54.025967 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697692544}
W20260709 13:10:54.034108 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 308118us
I20260709 13:10:54.034868 30300 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.026237 30274 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.038146 30275 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.038357 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.039062 29715 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:54.038949 30275 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.042407 30275 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.037163 30260 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:54.044265 30260 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.046345 30260 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.049654 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.050328 29008 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:54.052225 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.053509 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.054284 30289 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.054802 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.055408 29001 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:54.055498 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.056322 29010 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:54.057783 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.059535 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.060245 29005 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:54.030190 30290 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.063519 30277 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.062526 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.065430 30272 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.067575 30272 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.066524 30283 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.071336 30299 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:54.103572 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.077809 30087 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.123700 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:54.182451 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9f00 after lost signal to thread 27904
I20260709 13:10:54.071981 30277 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.124645 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.262436 30267 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.264233 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.288008 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.288722 29007 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.264958 30299 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.079919 29683 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:54.265522 30278 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.266176 30265 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.266904 30288 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.266935 30263 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.267524 30303 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.267472 30272 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.268450 30261 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.270623 30235 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.272837 30302 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.274055 30298 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.189810 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:54.071520 30283 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.279455 29079 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:54.279342 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:54.280107 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 155370us
I20260709 13:10:54.284454 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.286682 29076 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:54.301855 30265 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:10:54.298041 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 218194us
I20260709 13:10:54.298096 30278 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.302052 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.328846 30299 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.301687 30303 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:54.303472 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002a3ea0 after lost signal to thread 28942
I20260709 13:10:54.302443 30288 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.303905 30261 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.336298 30304 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.305874 30264 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:54.305922 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 240636us
I20260709 13:10:54.307624 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.307003 29078 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:54.310714 30263 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.312448 30283 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.295332 30307 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.313164 29005 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
W20260709 13:10:54.331694 28975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.842s	user 0.007s	sys 0.015s
I20260709 13:10:54.331910 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.332790 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.299293 30267 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.342286 29680 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:54.312877 30302 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.340541 30261 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.347183 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.348639 30270 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.330268 29051 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:54.334456 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:10:54.352551 28975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.008s	sys 0.015s
I20260709 13:10:54.308176 30235 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:54.353219 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7d80 after lost signal to thread 29034
I20260709 13:10:54.330530 30289 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.354351 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:54.354209 29002 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.336871 30304 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.334654 30278 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.358688 29051 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:54.363305 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:54.364874 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.365413 29712 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
W20260709 13:10:54.365551 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7520 after lost signal to thread 29109
I20260709 13:10:54.354358 30263 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:10:54.366024 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7500 after lost signal to thread 29110
I20260709 13:10:54.369354 30303 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.371212 29051 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:10:54.379333 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08003016a0 after lost signal to thread 29183
I20260709 13:10:54.382283 29051 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:10:54.388626 29738 debug-util.cc:398] Leaking SignalData structure 0x7b0800325900 after lost signal to thread 29741
W20260709 13:10:54.389219 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9be0 after lost signal to thread 30212
W20260709 13:10:54.389694 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002832a0 after lost signal to thread 30250
I20260709 13:10:54.362601 30264 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.406562 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.379118 30285 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.381178 30270 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.355640 30298 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:54.383745 30268 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.377583 30263 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.393541 30259 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.365203 30295 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.425005 30202 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.425242 30269 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.425625 29122 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:54.403247 30264 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:10:54.434940 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 83490.6us
I20260709 13:10:54.436439 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.415868 30285 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.443364 28907 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52674 (request call id 256) took 1413 ms. Trace:
I20260709 13:10:54.418489 30270 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.382948 30307 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.415124 30303 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:54.397979 30289 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.451599 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.452345 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.452550 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.462797 29080 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.451936 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.453289 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.453681 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.454270 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.442636 30261 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.427547 30259 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.431736 30202 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.437216 30264 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.449494 28907 rpcz_store.cc:276] 0709 13:10:52.923189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:53.240204 (+317015us) service_pool.cc:224] Handling call
0709 13:10:54.335765 (+1095561us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":980,"reported_tablets":1}
I20260709 13:10:54.448987 30285 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.442957 30269 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.456699 30270 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.471455 30299 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:54.471578 30272 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.471720 30275 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.472035 30283 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.381426 30278 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:10:54.472688 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.443226 30292 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.503118 30292 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.503692 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.469704 29082 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.504983 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.505118 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.455092 30255 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.478403 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.479075 30262 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.477046 30269 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.479108 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.476763 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.479801 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.493731 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.469259 30259 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.464741 30261 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.472596 30202 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.496785 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.457280 30260 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:10:54.479960 30310 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:10:54.502099 29001 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.505266 30292 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.456254 30277 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.479321 30267 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.505389 30288 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.498265 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.508869 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.522960 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.472769 30295 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.523530 29080 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.517444 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.473459 30303 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.512872 30304 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.505599 29076 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.515297 30261 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.523130 29084 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.523926 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.509187 30255 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.514117 30262 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:54.508713 30275 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.511677 30299 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.540989 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.539920 30260 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.544708 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.545429 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.522647 30283 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.556825 30295 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.522101 30272 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.538586 30262 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.564102 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.568915 29153 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.572307 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.535787 29683 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:54.573350 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.577556 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.577767 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.580823 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.571234 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.583974 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.583427 30287 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.577777 30303 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.589588 30255 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.589284 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.591019 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.588582 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.591722 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.588318 29010 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.593030 30299 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.594589 29006 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.597748 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.598241 29080 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:54.600845 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.600997 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.601392 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.602006 29081 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:54.602973 30298 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.603003 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.603814 29007 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.604157 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.604813 30275 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.605370 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:10:54.581645 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 50756.9us
I20260709 13:10:54.606014 30260 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.606469 29683 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:54.607097 30283 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.607420 30301 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.587724 30287 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.607909 30301 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.608258 30287 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.608520 30301 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.583802 30265 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.634402 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.635347 30268 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.635988 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.597102 30289 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.635900 30268 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.577338 30263 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.641037 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.640240 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.644220 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.644773 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.644769 30298 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:54.645632 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.655562 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.656684 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.657435 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.657739 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.658623 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.665699 29002 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.659504 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.659750 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.659814 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.664075 30235 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.664224 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.664755 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.671728 29010 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:54.687354 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.693674 30314 raft_consensus.cc:1064] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:54.653609 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.698622 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.704746 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.706915 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.707592 29154 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:54.708952 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.710013 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.709888 30314 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.710479 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.711222 29156 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:54.711297 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.713748 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.715294 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.672662 30298 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.682755 30302 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.706142 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.729483 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.730484 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.732612 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.687374 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:54.741449 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.750548 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.750694 29150 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:54.751185 29158 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:54.751935 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:10:54.752738 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63701.9us
I20260709 13:10:54.754671 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:54.740718 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.760648 30307 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:54.762209 30202 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.763381 29085 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:54.764679 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.771310 30264 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.772575 29716 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:54.774859 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.778802 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.745886 30272 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.804111 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.784176 30319 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.805317 30319 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.805521 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.805730 30319 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.807042 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.807600 29151 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:54.809602 30263 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.787618 30307 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.810479 30307 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.810844 30307 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.812089 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.813048 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.813781 29158 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:54.784552 30299 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.812624 29152 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:54.824518 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:54.824856 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:54.827003 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:54.829826 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.830065 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.830425 29153 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:54.831157 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.832973 29125 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:54.846436 30202 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:10:54.847010 30318 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:10:54.849557 30259 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.851152 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.852246 29077 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:54.853214 30275 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.853595 30275 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.853940 30275 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.851735 30259 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:10:54.784148 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 103631us
I20260709 13:10:54.856980 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:54.857892 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:54.856218 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 118270us
I20260709 13:10:54.825254 30299 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.859217 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:54.861508 30264 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:54.864208 30265 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:54.865912 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:54.866508 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:54.869902 30202 consensus_queue.cc:1059] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:54.876247 30260 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:54.844759 30235 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.844555 30272 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:54.884615 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.885475 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.885766 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:54.885969 28975 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.675s (SENT) [suppressed 4 similar messages]
I20260709 13:10:54.886112 30301 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.886634 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.887338 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.879899 30265 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.889559 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.892397 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:54.878355 30299 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.863798 30268 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.897549 30272 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.873190 30287 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.904075 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.890146 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.873027 30285 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.908984 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:54.948829 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.897686 30235 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:54.890311 30260 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:55.015909 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.020227 30307 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:55.021622 30269 consensus_meta.cc:287] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.003s
I20260709 13:10:55.020509 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.003419 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:55.022053 30269 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:54.999898 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:55.023202 29712 raft_consensus.cc:2393] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:10:55.023940 30323 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.221s	user 0.005s	sys 0.003s
I20260709 13:10:55.023892 30301 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.009816 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.033560 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.023011 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.049935 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.051136 29084 raft_consensus.cc:2393] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:54.909153 30265 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:54.916407 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:54.988382 30298 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:54.952212 30277 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.003816 30272 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.057533 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.058336 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.912959 29077 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:55.018383 30235 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.018157 30260 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:54.891048 30261 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:54.939455 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:54.916419 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:54.953209 30262 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:10:55.145416 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 227326us
I20260709 13:10:55.065105 29151 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:55.124703 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.074326 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.147758 29002 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:55.059060 29082 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.017443 29155 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:55.146360 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.115402 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.152278 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:55.124243 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.120793 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.154170 29150 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
W20260709 13:10:55.155131 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 269155us
I20260709 13:10:55.160689 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.161046 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.161940 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.164172 29152 raft_consensus.cc:2393] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:55.164915 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.166695 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.064406 30277 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:55.114423 30285 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:55.145417 30307 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.073977 29154 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:55.168011 30325 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.171051 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:10:55.113149 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52779us
I20260709 13:10:55.167686 30277 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.171980 29715 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.188077 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.186525 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.189023 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.178596 30325 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:55.197661 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.197733 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.202832 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.181213 30288 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.206059 29083 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.203872 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:55.199882 30270 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.006s
I20260709 13:10:55.179065 30328 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:10:55.218014 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.217883 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.219342 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.219262 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.224416 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.226136 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.211838 30270 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.228719 29717 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.232815 30270 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:55.217190 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.238880 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.240319 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.240691 30312 raft_consensus.cc:1064] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:55.241151 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.242650 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.244314 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.246507 30259 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:55.246887 30259 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:55.243294 30312 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.252255 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:55.210857 30275 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:55.262876 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.264407 29159 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.233659 30269 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:55.261677 30287 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:55.265715 30262 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.264431 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.247885 30259 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.274121 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.274935 30259 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.268900 29085 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:55.273488 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.275869 30283 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:10:55.281749 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.282850 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.283275 30321 consensus_queue.cc:1059] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:55.284013 29713 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.284344 30321 raft_consensus.cc:3055] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:55.286607 30321 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:55.287505 30321 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.288728 30321 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.290683 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.291998 30283 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:55.292438 30283 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:55.290843 30262 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.296787 30268 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:55.297905 30319 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.297927 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.300314 30319 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.300738 30283 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.301394 30303 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:55.304445 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.306721 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.302078 30283 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.311514 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.313915 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.311105 30262 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.315310 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:55.321633 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.326846 30328 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.328830 30275 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.329483 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.332415 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.333631 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.333940 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.333045 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.317194 30261 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.322045 30288 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:10:55.341095 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.341850 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.342252 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:55.344290 29713 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:10:55.345013 30288 raft_consensus.cc:3055] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:55.345443 30288 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:55.346309 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.346171 30272 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:55.352982 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.352728 29713 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:55.355432 29084 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:55.354562 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.366995 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.367218 29717 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:55.368376 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.368815 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.354265 30275 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:10:55.370791 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.370992 30275 raft_consensus.cc:3055] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:55.371379 30275 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:55.372351 30275 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.373726 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.374948 30275 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.376412 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:55.346318 30288 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.383862 29122 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.385504 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.386072 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.386425 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.386970 30287 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.388376 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.389197 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.389689 29158 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:55.390686 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.394862 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.397374 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.386096 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.389957 30261 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:55.393762 30328 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:55.412014 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.413847 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.418656 29007 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:55.399279 30329 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.419833 30329 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:55.420233 30329 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.396256 30299 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:55.422600 29156 raft_consensus.cc:1240] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.422789 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.409686 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:55.426496 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.401194 30202 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:10:55.410537 30328 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:55.422555 30325 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:10:55.443914 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 82407.7us
I20260709 13:10:55.445848 29713 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:10:55.446283 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.448145 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.449529 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.443919 30325 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:55.450162 29155 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:55.450393 30325 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.438974 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.435803 28908 catalog_manager.cc:5697] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:55.443498 29078 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:55.456611 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.459583 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.429505 30288 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.451524 30328 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:55.467986 30307 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.489259 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.459020 30267 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:10:55.494685 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.493594 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.488790 30273 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.489300 30307 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:55.505059 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:55.492885 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:55.514540 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.515654 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.495910 30313 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.494434 30299 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.515753 30287 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:55.517987 29154 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:55.503798 30273 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:55.523226 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:55.502794 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.509666 30307 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:55.529002 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.504776 30272 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.526834 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:55.529335 30287 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:55.532793 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.527380 30273 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.518620 30335 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.543161 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.544667 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.580691 29159 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:55.541473 30335 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:55.583073 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.568325 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:55.588701 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54813.7us
I20260709 13:10:55.588826 29154 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:10:55.592274 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:55.617686 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.585196 30335 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.621820 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.624663 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.632259 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.613214 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:55.635257 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 44434.2us
I20260709 13:10:55.567308 30287 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:10:55.644821 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 125701us
I20260709 13:10:55.649572 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:55.652714 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.655731 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:10:55.656149 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 89357.3us
I20260709 13:10:55.658772 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.660090 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.660533 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:55.661243 29159 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:55.661283 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:55.662192 29717 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.620004 30277 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.680228 29122 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:55.677402 30235 consensus_meta.cc:287] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
W20260709 13:10:55.692875 30265 consensus_meta.cc:287] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.572s	user 0.005s	sys 0.005s
I20260709 13:10:55.693347 30265 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:55.695410 30265 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.688184 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.696683 29153 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:55.680405 30299 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:55.698752 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.686241 30313 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.704300 30313 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:55.704704 30313 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.716187 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.716253 30334 consensus_queue.cc:1059] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:55.717662 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.719223 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.686810 30342 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:10:55.725711 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.725927 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.728864 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.729463 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:55.730393 29122 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.730159 30325 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:55.734314 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.736290 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.686185 30235 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:55.697978 30278 consensus_queue.cc:1059] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:10:55.730927 30267 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:55.745327 29122 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.750994 29156 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:55.754980 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:55.762369 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:55.738019 30327 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.741178 30340 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.714937 30329 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:55.742758 30275 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.769609 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:55.774171 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 65700us
I20260709 13:10:55.773450 30327 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:55.776367 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:55.782543 29152 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:55.778288 30337 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:55.780531 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:55.793601 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:55.771441 30260 consensus_meta.cc:287] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.610s	user 0.008s	sys 0.003s
W20260709 13:10:55.815070 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 47755.8us
I20260709 13:10:55.817473 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:10:55.819372 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 43899.3us
I20260709 13:10:55.795003 30327 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.822531 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:55.775466 30299 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.790400 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:55.795264 30337 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:55.812722 30340 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.830027 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.834405 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.772244 30298 consensus_queue.cc:1059] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:10:55.842849 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.845531 29122 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:55.821156 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:55.824200 30202 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:10:55.824388 30278 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:55.836100 30275 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:55.814863 30260 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.840938 30277 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:55.857417 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:55.781960 29682 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:55.846666 30312 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.846853 30283 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.853675 30299 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.859436 30321 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:55.896581 30235 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.899751 30329 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:55.900668 29151 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:55.922181 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:55.916761 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:55.924139 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 63636.3us
I20260709 13:10:55.901155 30321 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:55.914223 30319 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:55.903651 30289 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:10:55.936580 30321 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:55.926379 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 163822us
I20260709 13:10:55.965353 30337 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:55.978448 30317 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:55.980448 30317 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:55.926106 30273 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:55.990461 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 170112us
W20260709 13:10:55.952131 30303 consensus_meta.cc:287] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.649s	user 0.000s	sys 0.012s
W20260709 13:10:55.999315 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65060us
I20260709 13:10:56.027262 28975 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.026887 30317 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:56.027081 30313 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:55.979717 30307 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:56.038661 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 116355us
W20260709 13:10:56.046278 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 226109us
I20260709 13:10:56.048413 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.048435 30260 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:56.054957 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.057044 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.038923 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.061537 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:56.062727 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:56.063375 29150 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:56.049460 30348 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:56.051270 30337 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.063901 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:56.074666 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:56.064280 30288 consensus_meta.cc:287] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.003s
I20260709 13:10:56.077423 29152 raft_consensus.cc:2376] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2. Re-sending same reply.
I20260709 13:10:56.095460 29154 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:56.096194 30349 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.096865 30349 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.097929 30341 raft_consensus.cc:3055] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:56.066224 30348 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:56.087603 30287 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:56.102624 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 70174.5us
I20260709 13:10:56.110760 30348 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.117091 30356 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.142463 29157 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:56.109091 30341 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:56.129865 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:56.132174 29158 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:56.134771 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:56.136662 30314 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:56.146821 29717 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.084087 30353 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.082s
W20260709 13:10:56.153705 30335 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.000s
I20260709 13:10:56.154204 30314 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:56.146714 30273 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:56.154328 30313 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:56.173328 29682 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.175164 30341 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:56.162164 30335 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:56.180469 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.180984 30287 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.182475 30334 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:56.184916 30287 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.184607 30334 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:56.187340 30307 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:56.168643 30356 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:10:56.194456 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 48936.9us
W20260709 13:10:56.191993 30328 consensus_meta.cc:287] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.000s
I20260709 13:10:56.172240 30321 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:56.187126 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.328562 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.313077 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:10:56.288411 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 201330us
I20260709 13:10:56.307571 29085 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:56.319443 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.335976 29082 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.313964 30325 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:56.337211 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.337450 30341 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.186828 30334 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:56.199828 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 354313us
I20260709 13:10:56.341672 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.219766 30268 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:56.293063 29717 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:56.340405 30264 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.207965 30336 raft_consensus.cc:3055] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:56.350704 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.351217 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.207767 30202 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:56.340145 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.358932 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.359985 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.361367 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.243503 29010 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.352628 29122 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.353070 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:56.354208 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.224896 30335 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:56.363227 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.358053 30202 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:10:56.365654 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 338424us
I20260709 13:10:56.352420 30299 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.322554 30327 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:56.387625 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.388485 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.389604 29048 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:10:56.390394 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 333374us
I20260709 13:10:56.391993 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.392671 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.393512 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:56.393802 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:56.394038 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:56.394296 29158 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:56.357015 30336 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:56.417487 29150 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:56.318755 29008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
mode: GRACEFUL
new_leader_uuid: "243d4ae4a4ae48b2979e32874472d256"
 from {username='slave'} at 127.0.0.1:47940
I20260709 13:10:56.333891 30328 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:56.358987 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 170714us
I20260709 13:10:56.425135 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:56.417826 30348 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:56.422724 29008 raft_consensus.cc:606] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Received request to transfer leadership to TS 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.436093 30348 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:56.432969 29157 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:56.388736 30307 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.388638 30313 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:56.230940 30356 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:56.447649 29682 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.452528 30264 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.454669 30313 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:56.454314 30202 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:56.362743 30317 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:10:56.416384 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.463761 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.364789 30353 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.470432 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.471593 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.472317 29007 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:10:56.472244 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.473186 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.473306 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.473778 29002 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.390272 30366 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.426272 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.476056 30336 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:56.476671 29001 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:56.478883 30321 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:56.428634 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:56.201252 30314 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:56.482430 30327 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:56.482802 30314 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:56.483491 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 57297.4us
I20260709 13:10:56.366170 29077 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:56.484261 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.483526 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.438541 29151 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:56.397210 30289 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.439714 29153 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:56.388623 30360 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.486636 30348 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:56.440682 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.441313 30363 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:56.441766 30370 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:10:56.422940 30283 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.442448 30328 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:56.455621 30313 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:56.457322 29154 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:56.458536 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.434466 30268 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:56.466126 29006 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:56.487532 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.514221 29152 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:56.515395 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:56.491269 30336 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.491629 30334 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.492193 30337 raft_consensus.cc:1064] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:10:56.502974 29156 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:56.458395 30356 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:56.488605 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:56.516974 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 58533.8us
I20260709 13:10:56.526122 29712 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:56.527894 28976 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.406890 28975 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.521878 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.524305 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:10:56.528527 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 125443us
I20260709 13:10:56.530488 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:56.532823 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:56.534245 29080 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:56.534636 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 147594us
I20260709 13:10:56.514089 30369 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.535358 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.517448 30268 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:56.541374 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.539179 29050 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.545280 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.550196 30371 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:56.530383 30328 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:56.550048 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:56.551796 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.551509 29050 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.549731 28975 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.557281 29008 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:56.535279 30337 raft_consensus.cc:1068] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Unable to promote non-voter ca45420e033847489ad989c1c6eed3ef: Service unavailable: leader transfer in progress
I20260709 13:10:56.557965 30357 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:56.558131 30373 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.542582 30363 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:56.569161 30363 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:56.569559 30363 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.548965 30317 consensus_queue.cc:1059] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:10:56.549472 30259 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:56.560132 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.544414 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:56.566817 29082 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:56.567253 29122 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.550587 30371 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:56.574082 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:10:56.555673 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 93324.6us
I20260709 13:10:56.563059 30342 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:56.553952 30264 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.537403 30325 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:10:56.593031 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.572225 30317 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:56.571935 29010 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:56.587987 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.563148 29081 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:10:56.608158 30285 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:10:56.609390 30285 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:56.609664 30202 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.609800 30285 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:56.610850 30285 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:56.607501 30289 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:56.580636 30299 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.597419 29122 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.598726 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.604359 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.580289 30362 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.596752 30355 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:56.586711 29002 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:56.607674 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:56.584278 30259 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:56.679886 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:56.578759 30376 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:56.577864 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.592478 30371 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.620597 30347 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.649017 30285 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:56.594532 30342 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:56.661159 29007 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.661026 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:56.661026 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:56.597065 30317 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:56.662636 30273 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.663995 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:56.664853 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:56.665371 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.612572 30327 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.605640 30366 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:10:56.597365 30264 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.669996 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.682773 29122 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.612305 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:10:56.683992 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 106235us
W20260709 13:10:56.795992 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 113310us
I20260709 13:10:56.796288 28975 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.681224 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:10:56.688584 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 296630us
I20260709 13:10:56.689275 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:56.690934 29078 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:56.691958 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:56.797719 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.682233 30348 raft_consensus.cc:2764] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:10:56.682574 30355 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:56.688023 30373 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:56.700834 29008 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:56.688311 30333 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:56.691953 30342 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.757908 30259 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:56.797020 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.666239 30287 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
W20260709 13:10:56.816349 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:56.848732 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51793.6us
I20260709 13:10:56.841666 29003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
mode: GRACEFUL
new_leader_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:47940
I20260709 13:10:56.831200 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.847615 30313 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:56.849802 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.850005 29003 raft_consensus.cc:606] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Received request to transfer leadership to TS ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.850512 29003 raft_consensus.cc:614] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:56.791694 30319 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.831154 30267 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:56.856523 30377 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.857359 30335 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.858445 30303 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:10:56.863902 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 182782us
I20260709 13:10:56.795738 30299 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:56.830682 30373 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:56.843324 30333 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:56.851047 30334 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:56.843776 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 170625us
I20260709 13:10:56.852504 30355 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:56.955193 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 166619us
I20260709 13:10:56.959354 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:56.794642 30317 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:56.869952 30313 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:56.879128 30349 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:56.971521 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:56.869318 30299 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:56.889060 30373 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:56.974843 29085 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:56.853693 30325 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.101s
I20260709 13:10:56.893082 30333 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:56.888164 30303 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:10:56.974009 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 176360us
I20260709 13:10:56.982429 29154 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:56.986922 30267 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:56.988499 29007 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:56.964356 30377 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:10:56.982511 30369 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:56.967007 30319 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:56.993273 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:57.004853 29151 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:10:57.008000 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 155180us
I20260709 13:10:56.984140 29082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "243d4ae4a4ae48b2979e32874472d256"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:56.989259 30356 raft_consensus.cc:2764] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:10:57.015635 28975 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:57.017572 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58338.5us
I20260709 13:10:56.980159 29682 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:56.994042 30202 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:56.992564 30267 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:56.991966 30384 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.030862 29082 raft_consensus.cc:606] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Received request to transfer leadership to TS 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.037297 29082 raft_consensus.cc:614] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:57.008639 30319 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.032748 30287 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:57.048810 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55649.1us
I20260709 13:10:57.033061 30386 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
W20260709 13:10:57.048556 30336 consensus_meta.cc:287] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.523s	user 0.006s	sys 0.005s
W20260709 13:10:57.068204 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52684.6us
I20260709 13:10:57.048949 30264 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:10:57.061365 30268 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.002s
I20260709 13:10:57.063697 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.047456 30371 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.057389 30299 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:57.083745 29009 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:57.070348 30287 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.056159 30303 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.088490 28975 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.044021 30385 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.045292 30313 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.061753 30391 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:57.087802 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 55465.7us
I20260709 13:10:57.035974 30384 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.085358 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.070724 30273 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:57.095351 29006 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:10:57.091264 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 84128.3us
I20260709 13:10:57.092635 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:57.086845 30382 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:57.083566 30260 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:57.103914 30397 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.089761 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 88) took 1064 ms. Trace:
I20260709 13:10:57.111083 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.104156 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.114454 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.087832 30268 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.120589 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.125985 30319 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.126946 30384 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:57.127560 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.128435 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.129182 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.130062 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.130658 29003 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:57.134135 30268 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.137653 30287 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:57.122887 30382 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.113514 29077 rpcz_store.cc:276] 0709 13:10:56.025021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.025254 (+   233us) service_pool.cc:224] Handling call
0709 13:10:56.025721 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:56.490472 (+464751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:57.072774 (+582302us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54000c8ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:10:57.088968 (+ 16194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760361216}
W20260709 13:10:57.142335 30363 consensus_meta.cc:287] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.000s
I20260709 13:10:57.142850 30363 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.102350 30342 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.105208 30273 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.149080 29002 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:10:57.150249 30273 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:57.151022 30388 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:57.151506 30388 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:57.151855 30388 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.151969 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.111167 30260 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.153198 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.154206 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.154465 30260 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:57.155148 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.157480 29008 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:10:57.162496 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:57.163079 29717 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:57.163231 29083 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
W20260709 13:10:57.163908 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.164575 29125 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:57.165709 29125 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:10:57.169238 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 58233.6us
I20260709 13:10:57.170106 28975 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.094640 30267 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.171383 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:57.171944 29152 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.172214 30389 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:57.172691 30389 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.161465 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:57.179260 29152 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:10:57.182757 29154 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.182798 30289 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:57.116072 30385 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:57.186651 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:57.179258 30260 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:57.187654 28975 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.188607 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:57.192281 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.192389 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:57.193920 30268 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:57.194583 30287 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:57.194990 30287 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.193964 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.195545 30287 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.196770 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.197505 29009 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:57.198793 29680 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:57.199795 30384 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:57.200271 30384 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.200709 30384 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.204649 29001 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.195600 30391 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.194939 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.209879 29008 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:10:57.206324 29084 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:10:57.206243 29009 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.207840 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.182410 30264 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:57.184000 30273 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:57.183911 30376 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:57.211699 28975 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.195266 30369 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:57.217473 28975 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.222245 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.214398 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:57.218235 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:57.189531 30356 raft_consensus.cc:2764] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:10:57.222280 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 180489us
I20260709 13:10:57.224280 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:57.224148 29001 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.224933 29122 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.225847 28975 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.227949 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.194609 30268 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:57.230495 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:57.233283 29157 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.233827 29150 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.234977 29151 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.213248 30336 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.180836 30382 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:10:57.194623 30389 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:57.212755 30319 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:10:57.211117 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 239810us
I20260709 13:10:57.212944 30273 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:57.240926 29156 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.251556 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:57.216153 30260 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:57.221547 30369 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:57.242959 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:57.229275 30347 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.227521 30337 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.255326 30398 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.255465 30395 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:57.256511 29154 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:57.256639 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.257433 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:57.258096 28908 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51350 (request call id 145) took 1839 ms. Trace:
I20260709 13:10:57.257858 30260 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.259354 30268 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.252290 30319 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.237228 30348 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.263808 29159 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:57.277760 30264 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.266225 30369 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.267511 30387 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.239074 30267 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.267541 30394 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:10:57.272117 30401 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:10:57.253505 30273 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.215543 30397 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:10:57.255885 30396 consensus_queue.cc:1059] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:10:57.235350 30393 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.283269 30358 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.283732 30390 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:57.281307 30373 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:57.281584 30395 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:57.284499 30367 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.288398 30363 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.275117 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:57.294342 30317 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.291846 30360 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.300148 30319 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:57.287477 30333 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.310525 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:57.242394 30388 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:57.345907 29010 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:10:57.328838 30360 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:57.332479 30399 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:57.332962 30367 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:57.322829 30395 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.333622 30390 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:57.328949 30358 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.336287 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.312328 30355 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.350800 30398 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:57.351585 30388 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.348133 30319 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.346757 30333 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.349274 30327 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:57.358654 30360 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.366881 30342 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.368662 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:57.321902 30397 consensus_queue.cc:1243] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:10:57.373027 30358 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:57.274070 28908 rpcz_store.cc:276] 0709 13:10:55.418463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:55.418579 (+   116us) service_pool.cc:224] Handling call
0709 13:10:57.258038 (+1839459us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:55.832600 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:56.028343 (+195743us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:56.033412 (+  5069us) write_op.cc:435] Acquired schema lock
0709 13:10:56.038481 (+  5069us) tablet.cc:662] Decoding operations
0709 13:10:56.098736 (+ 60255us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:56.117910 (+ 19174us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:56.121483 (+  3573us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:10:56.152661 (+ 31178us) tablet.cc:701] Row locks acquired
0709 13:10:56.152709 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:10:56.152866 (+   157us) write_op.cc:270] Start()
0709 13:10:56.153007 (+   141us) write_op.cc:276] Timestamp: P: 1783602656152813 usec, L: 0
0709 13:10:56.153042 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:10:56.153761 (+   719us) log.cc:844] Serialized 595 byte log entry
0709 13:10:56.349715 (+195954us) op_driver.cc:464] REPLICATION: finished
0709 13:10:56.592796 (+243081us) write_op.cc:301] APPLY: starting
0709 13:10:56.592936 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:56.593062 (+   126us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:56.593243 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:56.593546 (+   303us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:56.593836 (+   290us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:10:56.593900 (+    64us) write_op.cc:312] APPLY: finished
0709 13:10:56.656540 (+ 62640us) log.cc:844] Serialized 23 byte log entry
0709 13:10:56.690645 (+ 34105us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:56.695765 (+  5120us) write_op.cc:454] Released schema lock
0709 13:10:56.701909 (+  6144us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:57.059426 (+357517us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b54000f7c28. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411bbfd9e1 00007f411bbfc64c 00007f411928d3fd 00007f4119279389 00007f4119278b53 00007f411927cd61 00007f411927cd19 00007f411927cca9 00007f411927cc71 00007f411927be9d 000000000038f354
0709 13:10:57.059869 (+   443us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:57.059905 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":237648,"apply.run_cpu_time_us":14226,"apply.run_wall_time_us":472191,"num_ops":1,"prepare.queue_time_us":162404,"prepare.run_cpu_time_us":11954,"prepare.run_wall_time_us":501212,"raft.queue_time_us":32662,"raft.run_cpu_time_us":44215,"raft.run_wall_time_us":181800,"replication_time_us":196542,"spinlock_wait_cycles":2826496,"thread_start_us":421293,"threads_started":4,"wal-append.queue_time_us":159191}]]}
I20260709 13:10:57.377313 30399 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.368160 30390 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.463608 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.472203 30288 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:57.481556 30288 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.481596 30287 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.476706 29009 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
W20260709 13:10:57.479665 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 107233us
I20260709 13:10:57.484269 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.485347 29048 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:10:57.464287 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 86116.4us
I20260709 13:10:57.490216 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.491253 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:10:57.492149 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.492724 30289 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:57.493307 30289 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:57.494287 30402 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:10:57.493850 30289 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.494634 30288 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:57.496546 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.495677 30366 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.502486 30287 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:57.503523 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:57.504138 29153 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.483197 30397 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.505040 29156 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.477180 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:57.507104 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:57.490621 30371 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.473866 30319 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.520295 29159 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.495508 30314 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.525435 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.527195 29155 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:57.519182 30360 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.519372 30341 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:57.537781 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:57.524606 30382 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.530160 30366 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.544343 29007 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.555675 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:57.527803 30314 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.562335 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:57.563475 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:57.567427 29150 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:10:57.570732 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.572163 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:57.572763 29078 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
W20260709 13:10:57.558883 30400 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:10:57.574095 30406 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:57.574146 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:57.583040 30404 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.290s	user 0.002s	sys 0.000s
I20260709 13:10:57.545544 30341 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.584180 30356 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:57.584117 30392 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:10:57.586997 30356 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:57.587180 29156 raft_consensus.cc:1240] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.587802 30356 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.589090 30356 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.588387 29156 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.555135 30363 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:10:57.587684 30392 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.593266 30283 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:57.599216 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:57.605551 29078 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:57.582150 30382 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.611559 29008 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:57.619330 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1228) took 1042 ms. Trace:
I20260709 13:10:57.621276 29010 rpcz_store.cc:276] 0709 13:10:56.565965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.665065 (+ 99100us) service_pool.cc:224] Handling call
0709 13:10:57.608153 (+943088us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400291b54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:10:57.608295 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480617728}
I20260709 13:10:57.606767 30283 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.630365 29158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:10:57.632841 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:57.625955 28975 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:57.639427 29712 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:57.640002 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:57.639004 29158 raft_consensus.cc:606] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:10:57.634512 29717 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:57.604187 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:57.641914 29155 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:57.636812 30353 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.642692 30353 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:57.643561 30394 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.597337 30392 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:57.638257 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:57.584540 30337 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:10:57.644349 30333 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.652093 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:57.656611 30407 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:57.657186 30407 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:57.657337 30327 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:57.658399 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:57.659031 29010 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:57.627821 30355 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.661932 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:57.643605 30283 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:57.646209 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:57.669145 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1224) took 1104 ms. Trace:
I20260709 13:10:57.669672 29006 rpcz_store.cc:276] 0709 13:10:56.564600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.669684 (+105084us) service_pool.cc:224] Handling call
0709 13:10:57.668897 (+999213us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54001860d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:10:57.669074 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691880832}
I20260709 13:10:57.670883 29003 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.639400 30342 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:10:57.645097 30334 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:10:57.627024 30406 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.668798 30401 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:57.607761 30393 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:10:57.623078 30349 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:10:57.644099 30378 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.146s
I20260709 13:10:57.628271 30382 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.688057 30319 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:57.710104 30349 consensus_queue.cc:1243] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:10:57.709380 29153 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:57.690204 30401 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.695814 30342 consensus_queue.cc:1243] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:10:57.658115 30407 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.710784 30409 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:10:57.710479 30341 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:57.715904 29158 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.715986 29085 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:57.699162 30399 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.728832 29007 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:10:57.738593 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:57.738763 30409 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:57.739276 30409 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:57.739672 30273 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.741164 29150 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.737640 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:57.743912 30273 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:57.740396 30409 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.721005 30401 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.724486 30378 consensus_queue.cc:1243] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:10:57.752718 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:57.709355 30392 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.726708 30314 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:57.761358 29078 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:57.764748 29007 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:57.752022 30399 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:10:57.766072 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 231776us
I20260709 13:10:57.755290 30328 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:57.769495 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:57.759423 30289 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.782895 30264 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:10:57.785562 30367 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.788224 30367 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.798211 30397 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.776948 30399 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:57.799089 29076 raft_consensus.cc:1240] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:57.801429 30384 consensus_meta.cc:287] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.006s
I20260709 13:10:57.801918 30387 raft_consensus.cc:3055] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:57.801844 30384 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.802333 30387 raft_consensus.cc:740] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:57.779073 30314 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:57.804926 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:57.805624 29001 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.787258 30392 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.776813 30409 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:57.807310 29078 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.809650 29159 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:57.806958 30397 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:10:57.811352 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:57.812623 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:57.813870 29080 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:57.806816 30314 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:57.816661 30314 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:57.784300 30395 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:57.799698 30399 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.821228 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:57.819505 30303 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.818174 30410 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:57.830420 29001 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
W20260709 13:10:57.823541 30391 consensus_meta.cc:287] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.602s	user 0.004s	sys 0.008s
I20260709 13:10:57.834432 29680 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:57.831997 30303 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:57.803213 30387 consensus_queue.cc:260] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:57.831382 30384 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:57.835070 30391 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:57.838337 30369 consensus_meta.cc:287] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 13:10:57.837453 30319 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:57.836112 30410 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:57.856135 29079 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
W20260709 13:10:57.922170 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:57.968466 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:58.009449 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.082825 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:58.086462 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.854528 30387 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:58.090081 29050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.045s	sys 0.142s
W20260709 13:10:58.092300 29050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.045s	sys 0.142s
I20260709 13:10:58.064162 30383 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:58.098292 29080 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:58.132176 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.133014 29085 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:10:58.134536 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:58.135114 29159 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:58.136374 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:58.091777 30402 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:10:57.854189 30259 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.008s	sys 0.006s
I20260709 13:10:57.843566 29683 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:58.084609 30260 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:58.171417 29151 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:57.889873 30369 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.179020 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.184458 29159 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:10:58.185802 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:58.172405 30259 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.009s	sys 0.006s
I20260709 13:10:58.102658 30383 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:58.168134 30385 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.194406 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.208349 29085 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:58.209342 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.210573 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.194919 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:10:58.196732 29717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.021s	sys 0.068s
I20260709 13:10:58.202632 29154 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:58.135151 30327 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.174821 30321 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.207494 30319 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:58.226008 29004 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.238226 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:58.262190 30319 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.262569 30342 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:10:58.264550 30421 log.cc:927] Time spent T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:10:58.244768 30321 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.178264 30260 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.249519 30259 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:58.255671 30314 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.278813 30303 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.279891 30358 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.245646 30349 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:10:58.239075 30385 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.244957 30412 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:58.318966 30328 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:58.319468 30328 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.316838 29158 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:58.322839 30260 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.323544 30321 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.315699 30355 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.324380 30355 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:58.325187 30355 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.328961 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1679) took 1131 ms. Trace:
I20260709 13:10:58.329418 29004 rpcz_store.cc:276] 0709 13:10:57.197675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.335744 (+138069us) service_pool.cc:224] Handling call
0709 13:10:57.336022 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:58.247265 (+911243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.328700 (+ 81435us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54003e26d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:10:58.328896 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001153408}
I20260709 13:10:58.330174 30391 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.330585 30391 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:58.331467 30391 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.332588 30423 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.334897 29080 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:10:58.337438 30356 consensus_meta.cc:287] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.003s
I20260709 13:10:58.337809 30328 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.339027 30412 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:58.339064 30357 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.339596 30357 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:58.340030 30412 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.262249 30424 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.342242 29084 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.245970 30327 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.280928 30395 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.346261 29156 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:58.329432 30314 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.378192 30369 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.317101 30358 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.379703 30327 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:58.381042 30358 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:58.246917 30383 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.382026 30383 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.385489 30385 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:58.386276 30392 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.577s	user 0.005s	sys 0.002s
I20260709 13:10:58.387140 29153 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:58.307790 30259 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:58.362871 30424 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:10:58.347685 30401 consensus_meta.cc:287] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.000s
W20260709 13:10:58.402252 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 224450us
I20260709 13:10:58.413255 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:58.424651 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.340483 30357 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.399132 30424 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:10:58.420293 30367 consensus_meta.cc:287] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.632s	user 0.003s	sys 0.007s
W20260709 13:10:58.464299 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 270014us
I20260709 13:10:58.465322 29085 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:10:58.444268 30409 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.001s
I20260709 13:10:58.447084 30373 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.404673 30401 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.468165 30367 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:58.490949 30401 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:10:58.493291 30411 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.403s	user 0.000s	sys 0.005s
I20260709 13:10:58.493634 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:58.494257 29007 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.482929 30409 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.482726 30399 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:58.477890 30422 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:10:58.502880 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 259152us
I20260709 13:10:58.478438 30259 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.509238 29080 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:10:58.513505 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.518230 29081 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:58.496928 30409 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.519868 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:58.520978 29717 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.522164 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.501868 30399 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.527330 30407 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:58.530601 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.531338 29082 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:58.510197 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:58.502076 30367 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.544378 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:58.543387 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:58.544986 29712 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:58.539181 29717 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:58.554276 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.556286 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:58.520423 30426 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.565724 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:58.583676 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:58.584015 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.561244 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:58.553838 29709 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:58.585448 30328 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.586227 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.587397 29009 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.587767 29009 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:58.556407 30424 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:58.591284 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.592631 29002 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:58.558768 30409 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:58.553141 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 715267us
I20260709 13:10:58.591153 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:58.594187 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.594478 29009 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.594218 30433 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:58.582755 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.595779 29002 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:10:58.593156 29001 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.603605 29010 raft_consensus.cc:2393] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:10:58.603828 30387 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:58.604689 30371 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.608325 29082 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:58.555802 30303 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:10:58.610841 30432 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.611362 30432 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.612068 29007 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:10:58.612109 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 829) took 1005 ms. Trace:
I20260709 13:10:58.614624 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:58.614984 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:58.615487 30367 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.593358 30424 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:58.617062 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:58.618927 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:58.617033 30387 consensus_queue.cc:1243] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:10:58.618258 30373 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.620628 30387 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:58.613795 29077 rpcz_store.cc:276] 0709 13:10:57.606257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.610904 (+  4647us) service_pool.cc:224] Handling call
0709 13:10:58.611101 (+1000197us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:10:58.611386 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:58.611626 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:58.611679 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:58.611739 (+    60us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:10:58.611792 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.611873 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:58.612005 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37846912}
I20260709 13:10:58.609834 30319 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:58.622072 30319 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.608124 30433 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.621439 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1684) took 1260 ms. Trace:
I20260709 13:10:58.625502 29003 rpcz_store.cc:276] 0709 13:10:57.360837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.537247 (+176410us) service_pool.cc:224] Handling call
0709 13:10:57.670657 (+133410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:57.671347 (+   690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.621239 (+949892us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54001860d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:10:58.621361 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306012544}
I20260709 13:10:58.625133 30319 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.617802 30424 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.619252 30383 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:58.628226 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:58.623281 30362 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:58.626955 30433 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.629859 30353 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.632341 30383 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:10:58.633639 30353 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.636651 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.638247 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.638370 29010 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.638531 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1661) took 1400 ms. Trace:
I20260709 13:10:58.638797 29010 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:58.639348 30432 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.640686 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1676) took 1443 ms. Trace:
I20260709 13:10:58.641808 29002 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:10:58.642369 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.642714 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.638880 29003 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:10:58.640221 29005 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.639654 29010 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.644366 29010 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.641038 29008 rpcz_store.cc:276] 0709 13:10:57.196965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.224969 (+ 28004us) service_pool.cc:224] Handling call
0709 13:10:57.611330 (+386361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:57.620899 (+  9569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.640513 (+1019614us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400291b50. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:10:58.640633 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848829824}
I20260709 13:10:58.638996 29157 rpcz_store.cc:276] 0709 13:10:57.238188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.238332 (+   144us) service_pool.cc:224] Handling call
0709 13:10:57.239880 (+  1548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:57.464078 (+224198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:57.464107 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:57.467711 (+  3604us) log.cc:844] Serialized 26 byte log entry
0709 13:10:57.752661 (+284950us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:57.752714 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.492763 (+740049us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:10:58.638209 (+145446us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54003f66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:10:58.638300 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:58.638345 (+    45us) raft_consensus.cc:1685] finished
0709 13:10:58.638357 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:58.638478 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7406,"raft.queue_time_us":7695,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":303278720,"thread_start_us":253236,"threads_started":1,"wal-append.queue_time_us":358477}
I20260709 13:10:58.586328 30426 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.647377 30367 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.648070 30367 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.650230 30367 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.651059 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.653936 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.654254 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:58.654500 29008 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:58.655361 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.656448 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.656709 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 92) took 1965 ms. Trace:
I20260709 13:10:58.657002 29003 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:58.657239 29079 rpcz_store.cc:276] 0709 13:10:56.691585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.691750 (+   165us) service_pool.cc:224] Handling call
0709 13:10:56.692109 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:58.033968 (+1341859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.656422 (+622454us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000451d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:10:58.656631 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2671725952}
I20260709 13:10:58.659188 30348 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.659543 30348 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:58.659674 29003 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.655361 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.661738 29008 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
W20260709 13:10:58.668140 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 78985.4us
I20260709 13:10:58.656529 30406 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.668717 29007 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.669081 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:58.669907 29125 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:58.668202 30307 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.670665 30307 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.670926 29125 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:58.672617 30307 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:58.664453 30267 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.672654 30327 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:58.669999 30398 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:58.667742 30328 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.675563 29010 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:10:58.675478 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:10:58.677601 29717 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.634073 30383 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:10:58.684075 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.670959 30389 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:58.695919 29717 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
W20260709 13:10:58.696800 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.697655 30348 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.699077 30348 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.692303 30367 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.699265 29712 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:58.700543 30367 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:58.690987 30388 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:10:58.704231 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.704357 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:10:58.676019 30327 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:58.676357 30398 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.706024 30389 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:58.706142 30307 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:58.673945 30303 consensus_queue.cc:1059] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:10:58.673841 30267 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.706748 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:10:58.709491 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 49037.8us
I20260709 13:10:58.692325 30439 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.712527 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:58.690068 30328 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.707204 30327 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.708312 30398 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.715389 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:58.692904 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:58.720664 30319 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.720489 30424 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.719470 30341 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.721467 30267 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:58.713825 30307 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:58.722802 30307 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.723729 30424 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:58.724416 29001 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:10:58.720525 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43809.8us
W20260709 13:10:58.728962 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.746556 29716 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:10:58.742267 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.747407 29713 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: []
W20260709 13:10:58.760468 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.755834 30391 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.752759 30328 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:58.749411 30342 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:10:58.744422 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:58.718636 30367 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.746316 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:58.732398 30319 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:58.771261 30319 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.749619 29708 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:58.739261 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 833) took 1013 ms. Trace:
I20260709 13:10:58.775738 29713 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.776546 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:58.777953 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:58.778503 29079 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.780405 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:10:58.780886 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 65580.7us
I20260709 13:10:58.780938 29085 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.776630 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:58.783766 29078 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:58.784087 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65566.4us
I20260709 13:10:58.793126 29082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:10:58.793674 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.795488 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:58.798544 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:58.791110 30303 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.803747 30202 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:58.804303 30202 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.804651 30202 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.805365 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:58.806895 30303 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.803791 30335 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:58.809192 30335 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.810215 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:58.810930 29153 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:58.811156 30335 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.811887 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:58.813330 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:58.814635 29157 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.774389 29081 rpcz_store.cc:276] 0709 13:10:57.725840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.736182 (+ 10342us) service_pool.cc:224] Handling call
0709 13:10:58.739101 (+1002919us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54001adbd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:10:58.739206 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500993920}
I20260709 13:10:58.819988 29157 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:58.822540 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:10:58.750134 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:58.823560 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:58.824802 29155 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.842960 29080 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:58.843003 30396 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:58.844012 30397 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.843756 30396 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:58.844887 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:58.798103 30259 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:10:58.825579 29156 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.784840 30429 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.828528 29078 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:10:58.817386 30392 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.822341 29154 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:58.735198 30426 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:58.735745 30389 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.742687 30382 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:58.832345 30424 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:58.822103 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:10:58.833895 30358 raft_consensus.cc:993] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:10:58.838685 29085 raft_consensus.cc:2410] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:10:58.847882 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:58.852905 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:58.853331 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:58.857327 29159 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.863693 29156 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:58.846211 30353 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:58.870615 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:58.870967 28975 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:58.871248 29150 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:58.871976 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:58.876467 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:10:58.876935 29002 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:58.850551 30303 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:10:58.855566 30392 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.880656 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.818389 30367 consensus_queue.cc:1059] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:10:58.803428 30391 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:10:58.823719 30342 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:10:58.851251 29077 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
W20260709 13:10:58.938859 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 50254.8us
I20260709 13:10:58.939291 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:58.889169 29122 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:58.942226 29079 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
W20260709 13:10:58.851954 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61624.6us
I20260709 13:10:58.939880 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:58.834231 30202 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.878631 30439 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.845798 30396 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.860720 30349 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:10:58.944605 29081 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:58.868060 30353 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:10:58.946095 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:58.944108 30268 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.942405 30397 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.944059 30259 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.893875 30328 raft_consensus.cc:1081] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
W20260709 13:10:58.949997 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60906.5us
I20260709 13:10:58.855610 30429 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:58.953342 30288 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:58.963349 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 831) took 1253 ms. Trace:
I20260709 13:10:58.963145 30268 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.964718 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:58.965296 29009 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:58.954622 30385 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:10:58.976486 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.893361 30387 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:10:58.893527 30432 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:10:58.981962 30202 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.889711 30303 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:58.978031 30283 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:10:58.982710 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:10:58.983284 30442 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:58.983824 30264 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:58.984433 30267 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:58.985569 30273 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:59.008306 30319 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:10:58.991576 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.994942 29084 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:10:58.995561 30377 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:58.974203 30288 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.979801 30391 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:58.974146 29076 rpcz_store.cc:276] 0709 13:10:57.709747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.713709 (+  3962us) service_pool.cc:224] Handling call
0709 13:10:57.714057 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:57.799491 (+ 85434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.963087 (+1163596us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b540034d354. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:10:58.963291 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194958720}
I20260709 13:10:59.004554 30303 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:59.007685 30392 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:59.010164 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:58.984440 30268 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.987767 30385 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.002369 29004 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:58.969820 30397 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.016516 29009 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:59.026973 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:59.027566 29009 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:59.028926 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:10:58.975005 30259 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:59.025660 30385 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.032090 30268 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:59.015198 30314 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:59.038195 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:59.036701 30273 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:10:59.038995 30268 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.041985 30268 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.015712 30288 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.002703 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:10:59.045334 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53307.8us
I20260709 13:10:59.012393 30348 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:59.010648 30349 consensus_queue.cc:1243] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:10:59.011291 30429 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:58.999548 30353 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.049508 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.050863 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:59.051726 29076 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:10:59.053640 29159 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:59.019304 30377 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.063316 30349 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:10:59.036785 30337 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:10:59.023773 30435 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:59.056463 30348 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.074177 30348 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:59.062150 30373 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:10:59.058540 30407 raft_consensus.cc:3055] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:59.048223 30264 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:59.021588 30391 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:10:59.064924 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:59.082474 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:59.078776 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.081058 30353 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.063817 30349 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.080811 30407 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:10:59.097920 29004 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:10:59.090807 30382 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.092938 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:59.210593 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.172560 29157 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:59.212970 29682 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:59.109900 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:10:59.180145 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:59.221192 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.228614 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:59.126219 30449 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:59.160457 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:59.239424 29003 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:10:59.108276 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.121685 30395 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.095901 30288 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.159471 30327 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:59.121538 30426 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:10:59.254153 30455 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.087s	user 0.006s	sys 0.000s
I20260709 13:10:59.254707 30327 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:59.254441 30426 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.127820 30451 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:10:59.181267 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.158270 30314 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.168591 30391 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:59.250211 29076 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:10:59.181766 30448 log.cc:927] Time spent T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:10:59.252390 29159 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:59.257234 29006 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:59.263108 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.272871 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.273703 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.222277 30396 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.243927 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:10:59.255431 30451 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:59.284248 29158 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:59.130602 30377 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:59.241518 30453 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:59.267611 30327 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.268738 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:10:59.284060 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.302929 29153 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:10:59.226889 30202 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:10:59.234146 30348 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:10:59.221980 30382 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.129856 30407 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:59.286373 30414 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:59.227285 30398 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:59.287060 30268 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.287468 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:10:59.287591 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43791.6us
I20260709 13:10:59.287837 30371 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:59.232273 30457 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:10:59.269771 30314 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.296427 30401 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:10:59.271374 30369 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:59.305588 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:59.276291 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:10:59.275568 30391 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.256989 30395 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.237104 30449 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:59.307965 30396 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.309983 29683 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:59.344122 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:10:59.346469 29154 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:10:59.346513 29152 raft_consensus.cc:2368] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3.
I20260709 13:10:59.306715 30392 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.310041 30389 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:59.366163 29157 raft_consensus.cc:2393] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:10:59.367218 29153 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:59.312988 30426 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:10:59.311297 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.341075 30349 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:59.309441 30327 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:10:59.311164 30202 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:10:59.347795 30407 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.372848 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:10:59.715263 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:10:59.387758 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43724us
I20260709 13:11:00.278129 29006 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:10:59.606902 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 294597us
W20260709 13:10:59.470587 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.368417 30461 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:10:59.429270 30389 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:00.283049 29001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1253) took 1618 ms (client timeout 1697 ms). Trace:
I20260709 13:10:59.365942 30371 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:10:59.365511 30369 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:10:59.503624 30450 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.127s	user 0.000s	sys 0.009s
I20260709 13:10:59.396601 29154 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:00.283495 29001 rpcz_store.cc:269] 0709 13:10:58.636147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.636246 (+    99us) service_pool.cc:224] Handling call
0709 13:11:00.254232 (+1617986us) spinlock_profiling.cc:243] Waited 87 ms on lock 0x7b5400311ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:00.254449 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200020864}
I20260709 13:10:59.415403 30398 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:59.399328 30439 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:00.292302 30389 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.389449 30429 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:59.397854 30460 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:59.496086 30454 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:10:59.462400 30356 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.838s
I20260709 13:10:59.372740 30288 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:00.306947 28906 catalog_manager.cc:5697] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:10:59.501652 30399 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.309963 30399 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.310420 30399 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:00.311600 30385 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.271s	user 0.010s	sys 0.001s
I20260709 13:10:59.496575 30382 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:59.499462 30451 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:00.267751 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 979031us
I20260709 13:10:59.687377 30319 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.368878 30401 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:00.256834 30314 consensus_meta.cc:287] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.912s	user 0.004s	sys 0.006s
I20260709 13:10:59.461593 30268 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:59.371959 30391 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:11:00.300647 30439 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:00.317554 30355 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:59.448982 30392 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.317708 30410 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:00.326016 30353 consensus_meta.cc:287] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.241s	user 0.011s	sys 0.000s
I20260709 13:11:00.298303 29050 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:00.309078 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1693) took 1041 ms. Trace:
I20260709 13:11:00.311975 30385 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.329818 30259 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:00.333214 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1679) took 1463 ms. Trace:
I20260709 13:11:00.318487 30327 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:00.313808 30429 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.348955 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:59.673213 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:59.958220 28905 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52674 (request call id 257) took 4341 ms. Trace:
W20260709 13:11:00.094107 30459 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.697s	user 0.001s	sys 0.001s
W20260709 13:11:00.362104 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:00.363550 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 65329.8us
W20260709 13:11:00.414839 29048 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.009s	sys 0.041s
I20260709 13:11:00.395853 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:10:59.612735 29711 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:59.769322 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.421202 29150 rpcz_store.cc:276] 0709 13:10:58.869741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.872895 (+  3154us) service_pool.cc:224] Handling call
0709 13:10:58.873143 (+   248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:58.873331 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:58.873356 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:58.889676 (+ 16320us) log.cc:844] Serialized 26 byte log entry
0709 13:10:59.016545 (+126869us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:59.016579 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:59.016640 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:00.333007 (+1316367us) raft_consensus.cc:1685] finished
0709 13:11:00.333025 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:00.333160 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432,"thread_start_us":52661,"threads_started":1,"wal-append.queue_time_us":133444}
I20260709 13:10:59.947557 30432 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:59.767628 30395 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.134974 30414 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.459944 29008 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:00.461983 29152 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:00.474709 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:00.471289 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.476346 29009 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
W20260709 13:11:00.476615 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.477381 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:00.473177 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.473639 29050 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:00.461730 29048 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.009s	sys 0.042s
W20260709 13:11:00.444553 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 79330.3us
W20260709 13:11:00.350217 30362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.221s	user 0.000s	sys 0.021s
I20260709 13:10:59.637642 30202 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:00.479629 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:00.331936 30319 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.374194 30268 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:10:59.617802 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:10:59.623171 30433 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:11:00.480554 29122 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.697s (SENT) [suppressed 32 similar messages]
I20260709 13:11:00.335494 30314 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:10:59.962870 30357 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:00.484530 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:00.484328 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 205039us
W20260709 13:11:00.481633 30362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.352s	user 0.001s	sys 0.021s
I20260709 13:11:00.512081 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:00.509948 30433 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:00.513638 30433 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.514577 30396 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.515089 30396 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.515487 30337 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:00.511790 30471 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:11:00.518383 30471 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.519332 28975 leader_election.cc:400] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3.
I20260709 13:11:00.471851 30399 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:10:59.913079 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 843) took 1130 ms. Trace:
I20260709 13:11:00.473768 28905 rpcz_store.cc:276] 0709 13:10:55.616948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:55.617117 (+   169us) service_pool.cc:224] Handling call
0709 13:10:59.957981 (+4340864us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2166,"reported_tablets":5,"spinlock_wait_cycles":307584}
I20260709 13:11:00.525815 29077 rpcz_store.cc:276] 0709 13:10:58.782728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.795176 (+ 12448us) service_pool.cc:224] Handling call
0709 13:10:59.893968 (+1098792us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b5400199e54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:10:59.906185 (+ 12217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104870144}
I20260709 13:11:00.423856 30412 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:00.482483 29008 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:10:59.934636 30449 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:10:59.685515 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:00.537303 30383 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.538226 29708 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:00.488960 30392 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:00.491983 30452 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.539353 30452 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.539758 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:00.492825 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 254) took 1553 ms. Trace:
W20260709 13:11:00.493610 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.493877 29076 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:00.493916 30202 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:00.470652 30348 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.495929 30395 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.496577 30342 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.497102 30373 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:00.485544 30429 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.497699 30447 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.351912 30371 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.486732 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:00.483620 30353 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:00.495185 30391 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.504953 30401 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:00.468302 29001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:11:00.486454 30460 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:00.471431 30410 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:00.484144 30461 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.095801 30413 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:00.466080 30356 consensus_queue.cc:1243] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:00.334996 30369 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.519624 28975 leader_election.cc:403] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:00.465279 30432 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:00.485456 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:00.569932 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 85122us
I20260709 13:11:00.515547 30396 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.518795 30471 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.375126 29154 rpcz_store.cc:276] 0709 13:10:59.266876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.373004 (+106128us) service_pool.cc:224] Handling call
0709 13:10:59.387444 (+ 14440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:00.299670 (+912226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.307892 (+  8222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:00.536808 30412 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.538373 30383 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.492813 30426 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.593650 30426 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.594267 30391 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.594650 30412 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.594720 30391 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.595667 30426 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:00.524837 30464 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.592404 30452 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.596544 30452 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.596323 30464 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.596971 30452 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.597538 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1267) took 1599 ms. Trace:
I20260709 13:11:00.598956 30452 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.597908 29004 rpcz_store.cc:276] 0709 13:10:58.997801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.130855 (+133054us) service_pool.cc:224] Handling call
0709 13:10:59.157459 (+ 26604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:59.416134 (+258675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:59.416169 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:59.416685 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:11:00.140888 (+724203us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:00.269018 (+128130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.597136 (+328118us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54003aaa54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:00.597191 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:00.597234 (+    43us) raft_consensus.cc:1685] finished
0709 13:11:00.597250 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:00.597480 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1005468,"raft.queue_time_us":28165,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":22794,"spinlock_wait_cycles":531917184,"thread_start_us":355320,"threads_started":1,"wal-append.queue_time_us":1093707}
I20260709 13:11:00.546116 29008 raft_consensus.cc:2393] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:00.519933 29007 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:00.525915 30362 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.550981 30259 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.503906 30389 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.551157 30449 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.487231 30414 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.537227 30468 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.531796 30466 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:00.557085 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45090.7us
I20260709 13:11:00.551337 30433 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:00.544664 29711 rpcz_store.cc:276] 0709 13:10:58.939250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.611905 (+672655us) service_pool.cc:224] Handling call
0709 13:10:59.612435 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:00.481592 (+869157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.492769 (+ 11177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:00.561812 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 75145us
I20260709 13:11:00.562623 29001 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:00.652469 29001 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:00.551563 30202 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:00.553392 30392 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:00.569049 30413 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.570070 29009 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:10:59.619980 30397 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.572116 30429 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.684527 30397 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:00.685309 30401 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.685421 30472 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.571830 30432 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.685813 30401 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.686004 30472 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.686650 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:00.687300 29708 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:00.570436 30461 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.687978 30401 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:00.688385 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:00.689061 30469 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.689432 29713 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:11:00.689575 30469 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.691121 30461 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.691968 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:00.693586 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:00.694000 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:00.694195 29710 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:00.694896 30401 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:00.695209 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:00.695645 30401 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.696345 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:00.696758 30401 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:00.696899 29717 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:00.575870 29082 raft_consensus.cc:606] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:00.697779 29082 raft_consensus.cc:614] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:00.565408 30463 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:00.698365 30463 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.698571 29083 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:00.569195 30460 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.701480 30460 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:00.558766 30395 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.704934 30472 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.691768 30469 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.705981 30395 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.560494 30373 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.706888 29009 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:00.707652 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:00.559602 30342 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.563174 30447 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.585163 30473 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:00.586563 30471 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.588712 30319 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.589277 30385 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.590029 30314 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.557411 30348 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.624255 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.628485 30487 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:00.629458 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.630721 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.755300 29152 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:00.630951 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.632100 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.636521 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.637823 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.757437 29157 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:00.638677 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:00.644330 29007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1260) took 1758 ms (client timeout 2012 ms). Trace:
I20260709 13:11:00.648074 30452 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.725320 30412 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.596552 30383 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.643379 30268 raft_consensus.cc:3055] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:00.789108 30268 raft_consensus.cc:740] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:00.666612 29078 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:00.667372 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:00.623324 30426 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.698740 30463 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:00.578670 29010 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:00.708695 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:00.713688 29001 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:00.720269 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.720439 29680 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:00.711545 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:00.727016 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:00.729148 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:00.744125 29150 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:00.745059 30283 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.709954 30410 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3.
I20260709 13:11:00.644073 30362 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.682900 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.657502 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:00.646311 30449 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.646688 30259 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.730165 30313 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.739786 30397 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.727725 30461 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:00.732270 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:00.739282 30355 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.741761 30475 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.729976 30464 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:00.739409 30407 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:00.650585 30468 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:00.709071 30392 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.710263 30483 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:11:00.664587 30466 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.789453 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:00.990413 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 509837us
I20260709 13:11:01.114688 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:01.114296 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.115634 29717 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:00.882472 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.116228 29157 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:00.875979 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.117007 29153 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
W20260709 13:11:01.011059 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:01.118078 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.790632 29682 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:00.743160 30460 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:01.101349 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.881322 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.857736 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1229) took 2344 ms. Trace:
W20260709 13:11:01.119194 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 328622us
W20260709 13:11:01.130713 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:01.131407 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 402276us
W20260709 13:11:01.133567 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 413183us
W20260709 13:11:01.115657 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.118077 29009 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:01.118913 29081 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:01.146783 30490 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:01.211153 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.166708 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:00.759346 30273 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.487s
I20260709 13:11:01.211899 29001 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.350368 29085 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:01.210412 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:01.199575 29717 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:01.264117 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.348325 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.272161 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.345736 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:01.355690 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:01.212339 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:01.350668 29083 rpcz_store.cc:276] 0709 13:10:58.513024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.555434 (+ 42410us) service_pool.cc:224] Handling call
0709 13:10:58.587890 (+ 32456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:00.699164 (+2111274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.857350 (+158186us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b5400146c54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:00.857612 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4852838784}
I20260709 13:11:01.372790 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:01.373111 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.352262 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:00.931023 30337 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:00.906155 30389 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:00.779520 30487 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:00.895990 30401 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.153337 30397 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.351197 29156 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:01.215478 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:01.367503 29048 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.003s	sys 0.021s
I20260709 13:11:01.404812 29078 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:00.904814 30484 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.469s
I20260709 13:11:01.374512 29084 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:01.374869 29009 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:01.113467 30357 consensus_meta.cc:287] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.000s
I20260709 13:11:01.415467 30397 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.374967 29153 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.416399 29002 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
W20260709 13:11:00.790612 30477 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:11:00.795215 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.417888 30391 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.376596 30490 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.418728 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:01.419577 29155 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:00.782620 30482 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.420595 30410 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:01.272711 30491 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:11:00.868505 30451 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.119612 30328 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:01.432883 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1698) took 2677 ms. Trace:
I20260709 13:11:01.381215 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.429539 29153 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:01.381556 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:00.780180 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:00.787652 30373 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:00.790138 30268 consensus_queue.cc:260] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.109644 30414 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.397089 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.460322 29151 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:01.046335 30473 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.867731 30371 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:01.210655 30283 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.119815 30341 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:01.148872 30429 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:00.860749 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42002 (request call id 8) took 2081 ms. Trace:
W20260709 13:11:01.404953 29048 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.003s	sys 0.021s
I20260709 13:11:01.111158 30395 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:01.349051 30432 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.405826 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:01.358243 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.793829 30267 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:01.269217 29708 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:01.210767 30383 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:01.107239 30413 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.260094 30288 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:01.374218 30382 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.013s	sys 0.018s
I20260709 13:11:01.210086 30479 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:01.380970 29154 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:01.411188 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.149165 30475 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.414139 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.414711 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.415261 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.415365 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:01.104861 29005 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.003s	sys 0.011s
I20260709 13:11:01.301945 30447 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.421276 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:00.799115 30412 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.219858 29077 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:00.871325 30439 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.420462 30490 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.434160 29001 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:01.434834 29085 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
W20260709 13:11:01.381080 29051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.007s	sys 0.024s
I20260709 13:11:01.154060 30313 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:00.764096 29007 rpcz_store.cc:269] 0709 13:10:58.875442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.884941 (+  9499us) service_pool.cc:224] Handling call
0709 13:10:59.617567 (+732626us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:00.633816 (+1016249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7830400}
I20260709 13:11:01.396672 30369 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:00.784775 30452 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.094995 30349 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:01.436612 29712 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:01.433331 29010 rpcz_store.cc:276] 0709 13:10:58.746656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.752636 (+  5980us) service_pool.cc:224] Handling call
0709 13:10:58.752952 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.131087 (+2378135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.424313 (+293226us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b540012f054. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:01.424470 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4197678848}
I20260709 13:11:01.436388 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.435479 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.153852 30461 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.420416 30482 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.420449 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.219564 30407 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.412724 30389 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:01.153523 30355 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.210505 30426 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.209964 30460 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.466732 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:01.424983 30337 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:01.470754 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 96702.4us
I20260709 13:11:00.906306 30202 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.474804 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:01.479389 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 765111us
W20260709 13:11:01.480324 29002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1261) took 2595 ms (client timeout 3361 ms). Trace:
I20260709 13:11:01.435729 30497 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:01.421074 30410 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.458564 30335 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:11:01.511195 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.460943 29076 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:01.485060 28975 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:01.492125 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.471794 29710 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:01.449510 30451 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.476238 30499 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:01.536497 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 69827.8us
I20260709 13:11:01.481180 30268 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:01.455847 30328 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.512007 29010 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:01.518776 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:01.524652 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.513068 29084 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:11:01.477614 30500 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:01.513418 29002 rpcz_store.cc:269] 0709 13:10:58.880289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.880433 (+   144us) service_pool.cc:224] Handling call
0709 13:11:01.475557 (+2595124us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b54001851d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:01.475665 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5813548800}
W20260709 13:11:01.513479 29005 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.008s	sys 0.015s
I20260709 13:11:01.513670 29009 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:01.556696 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:01.556854 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.557511 29081 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:01.511999 30479 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:01.539634 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.563877 29158 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:01.539530 30413 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:01.481873 30395 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:01.522516 29682 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:01.522927 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:01.512689 29006 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:01.589512 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1276) took 1122 ms. Trace:
I20260709 13:11:01.531477 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.590054 29002 rpcz_store.cc:276] 0709 13:11:00.466778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.569377 (+1102599us) service_pool.cc:224] Handling call
0709 13:11:01.589062 (+ 19685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.589218 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.589245 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.589285 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:01.589315 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.589365 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.589462 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:01.590927 29085 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:01.533735 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.512401 29151 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
W20260709 13:11:01.591761 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 73063.3us
I20260709 13:11:01.592211 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:01.513578 29051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.008s	sys 0.024s
W20260709 13:11:01.593271 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 839482us
I20260709 13:11:01.594394 29051 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:01.490470 30383 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.551538 29159 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:01.553009 29079 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.551405 29080 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:01.550452 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.477756 30473 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:11:01.562983 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:01.520525 30382 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.018s	sys 0.023s
I20260709 13:11:01.476291 30371 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.568688 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.530925 30327 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:11:01.496959 30267 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.480118 30341 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.530608 30337 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.541908 30480 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.530444 30497 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.562620 30396 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.636265 30383 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:01.588382 29152 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:01.579790 30449 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.561131 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.550756 30483 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:01.617398 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 94934.8us
I20260709 13:11:01.659051 29150 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:01.626567 29083 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
W20260709 13:11:01.617398 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60766.8us
I20260709 13:11:01.597350 30373 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:01.593659 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1279) took 1126 ms. Trace:
I20260709 13:11:01.636775 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:01.548287 30504 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:01.568804 30500 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.530898 30288 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.645376 30414 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:01.638221 29010 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.640473 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1277) took 1173 ms. Trace:
I20260709 13:11:01.640857 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.636586 29081 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:01.563627 30439 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:01.570021 30479 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.587721 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.673043 29007 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:01.661154 30480 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.623930 30353 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:11:01.624013 30447 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:01.645540 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 122689us
I20260709 13:11:01.623714 30314 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.646554 29051 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:01.591492 30490 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:01.582280 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.649629 29001 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:01.711261 29010 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:01.479192 29082 rpcz_store.cc:276] 0709 13:10:58.779595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.783223 (+  3628us) service_pool.cc:224] Handling call
0709 13:11:00.860452 (+2077229us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b5400146c54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583b853 00007f411a80960a 00007f4113d8f7d4 00007f4113d8f762 00007f4113d8f691 00007f4113d8f60c 00007f4113d8e8c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:00.860612 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4097815296}
I20260709 13:11:01.712179 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1282) took 1242 ms. Trace:
I20260709 13:11:01.712735 29010 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:01.635213 30495 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.583389 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.712603 29007 rpcz_store.cc:276] 0709 13:11:00.469629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.673812 (+1204183us) service_pool.cc:224] Handling call
0709 13:11:01.674105 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.674286 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.674317 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.674367 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:01.674412 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.711948 (+ 37536us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.712124 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:01.713613 29154 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:11:01.713487 30495 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.714607 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.714741 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 82) took 1086 ms. Trace:
W20260709 13:11:01.588217 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.549281 30499 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.715219 29007 rpcz_store.cc:276] 0709 13:11:00.628193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.713941 (+1085748us) service_pool.cc:224] Handling call
0709 13:11:01.714234 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.714394 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.714424 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.714482 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:01.714517 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.714575 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.714685 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:01.717348 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 83) took 1088 ms. Trace:
I20260709 13:11:01.717746 29007 rpcz_store.cc:276] 0709 13:11:00.628514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.716593 (+1088079us) service_pool.cc:224] Handling call
0709 13:11:01.716868 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.717035 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.717063 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.717107 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:01.717139 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.717193 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.717295 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:01.663875 29151 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:01.718870 29151 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:01.719214 30501 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:11:01.719877 30371 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.719688 29151 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.720633 29151 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.663638 29002 rpcz_store.cc:276] 0709 13:11:00.467572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.591162 (+1123590us) service_pool.cc:224] Handling call
0709 13:11:01.593065 (+  1903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.593269 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.593308 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.593362 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:01.593403 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.593473 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.593597 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:01.646628 30464 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:11:01.665207 29003 rpcz_store.cc:276] 0709 13:11:00.467068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.568417 (+1101349us) service_pool.cc:224] Handling call
0709 13:11:01.639927 (+ 71510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.640107 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.640134 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.640183 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:01.640225 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.640294 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.640409 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:01.670200 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.644191 30503 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:01.659896 30487 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.657889 29077 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:01.673811 30407 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:01.533178 30349 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.648202 30357 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:01.646395 30413 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:01.652792 30468 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.711362 29079 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:01.581241 30466 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.718292 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:01.664479 30373 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.652268 30259 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.663424 29680 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:01.722711 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1285) took 1229 ms. Trace:
I20260709 13:11:01.669034 30483 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.722633 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:01.644232 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 103312us
W20260709 13:11:01.727294 30498 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:11:01.801683 30460 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.660385 30481 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.727694 29002 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:01.671810 30401 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.670843 30414 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:01.962734 30481 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.963243 30414 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:01.963297 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 84) took 1274 ms. Trace:
I20260709 13:11:01.963752 29002 rpcz_store.cc:276] 0709 13:11:00.688546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.727098 (+1038552us) service_pool.cc:224] Handling call
0709 13:11:01.727392 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.963040 (+235648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.963241 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:01.664357 30475 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.964807 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.965149 30481 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.965412 29002 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:11:01.966074 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.966257 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.966323 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 250) took 1254 ms. Trace:
W20260709 13:11:01.928354 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.722075 30490 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.966862 29002 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:01.870563 30413 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:01.966823 29711 rpcz_store.cc:276] 0709 13:11:00.712352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.725838 (+ 13486us) service_pool.cc:224] Handling call
0709 13:11:00.909004 (+183166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.384804 (+475800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.397622 (+ 12818us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.410046 (+ 12424us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:01.421306 (+ 11260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.966073 (+544767us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54002fe4d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:01.966123 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.966250 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":883384064}
I20260709 13:11:01.968321 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1274) took 1502 ms. Trace:
I20260709 13:11:01.802598 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1247) took 1061 ms. Trace:
I20260709 13:11:01.969269 29077 rpcz_store.cc:276] 0709 13:11:00.740775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.563010 (+822235us) service_pool.cc:224] Handling call
0709 13:11:01.599821 (+ 36811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.731739 (+131918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.788288 (+ 56549us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:01.968721 29004 rpcz_store.cc:276] 0709 13:11:00.466241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.599776 (+133535us) service_pool.cc:224] Handling call
0709 13:11:00.666539 (+ 66763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:00.723894 (+ 57355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:00.723926 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:00.724461 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:11:01.045563 (+321102us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:01.045631 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.045869 (+   238us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:01.968113 (+922244us) raft_consensus.cc:1685] finished
0709 13:11:01.968140 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.968266 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25248,"raft.queue_time_us":1662,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":16196,"thread_start_us":320657,"threads_started":1,"wal-append.queue_time_us":810308}
I20260709 13:11:01.966674 29083 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:01.970635 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.971076 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.971323 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 90) took 1277 ms. Trace:
I20260709 13:11:01.801189 30497 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.732725 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.971932 29004 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:01.973312 30349 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.974159 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:01.974684 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1278) took 1507 ms. Trace:
I20260709 13:11:01.797211 28976 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:01.729907 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 83418.5us
I20260709 13:11:01.948932 30356 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:01.955127 30426 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:01.957813 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 87) took 1327 ms. Trace:
I20260709 13:11:01.957461 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1273) took 1491 ms. Trace:
I20260709 13:11:01.806890 30342 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.958382 30328 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.800647 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.958968 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.801019 30494 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:01.715770 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.807039 30407 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.159s
I20260709 13:11:01.801565 30452 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.960551 29010 rpcz_store.cc:276] 0709 13:11:00.492915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.712146 (+1219231us) service_pool.cc:224] Handling call
0709 13:11:01.712441 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.713266 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.722659 (+  9393us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:01.961186 29080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 850) took 1304 ms (client timeout 1517 ms). Trace:
I20260709 13:11:01.784691 30451 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:01.961897 30509 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:01.770303 30357 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:01.937470 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.971165 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:01.849471 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:01.977610 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.664172 30504 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:01.975049 29006 rpcz_store.cc:276] 0709 13:11:00.467316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.590103 (+1122787us) service_pool.cc:224] Handling call
0709 13:11:01.590411 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.967697 (+377286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.967728 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.967770 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:01.967799 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.974479 (+  6680us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b540035e3d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:01.974522 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.974631 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":840992128}
I20260709 13:11:01.953843 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:11:01.742583 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.432418 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:01.961627 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 298220us
W20260709 13:11:02.257601 29151 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.001s
I20260709 13:11:01.706009 30447 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:02.441105 29152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 948) took 1739 ms (client timeout 1907 ms). Trace:
W20260709 13:11:02.498272 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65972.9us
W20260709 13:11:02.480487 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 683312us
I20260709 13:11:02.011445 29156 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:02.481483 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.499028 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:01.820982 30433 consensus_meta.cc:287] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.169s	user 0.007s	sys 0.005s
I20260709 13:11:02.096006 29081 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:02.500984 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:02.500384 29010 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
W20260709 13:11:01.980069 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 257464us
I20260709 13:11:01.971751 29150 rpcz_store.cc:276] 0709 13:11:00.694276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.150010 (+455734us) service_pool.cc:224] Handling call
0709 13:11:01.971166 (+821156us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54002a35d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:01.971273 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":661374208}
I20260709 13:11:02.431682 29051 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:02.497088 29680 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:02.436236 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1244) took 1764 ms. Trace:
W20260709 13:11:02.210939 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.657862 30506 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:02.507537 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:02.514281 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.515455 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:02.496994 29151 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:02.516696 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.516906 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:02.498721 29152 rpcz_store.cc:269] 0709 13:11:00.701633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.295125 (+593492us) service_pool.cc:224] Handling call
0709 13:11:02.440892 (+1145767us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54003babd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:02.441035 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493294336}
I20260709 13:11:02.431584 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.979746 29159 rpcz_store.cc:276] 0709 13:11:00.630637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.630748 (+   111us) service_pool.cc:224] Handling call
0709 13:11:01.957576 (+1326828us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b54002b52d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:01.957711 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1828204928}
I20260709 13:11:02.506897 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:02.511130 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 79548.9us
I20260709 13:11:02.507076 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:01.801074 30495 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:02.172261 30327 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.005s	sys 0.024s
I20260709 13:11:01.786244 30412 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.844931 30499 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.508114 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.571121 29156 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
W20260709 13:11:02.510061 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.507769 29078 rpcz_store.cc:276] 0709 13:11:00.671718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.558133 (+886415us) service_pool.cc:224] Handling call
0709 13:11:01.558342 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.717052 (+158710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.717079 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.717112 (+    33us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:01.717139 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.435936 (+718797us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54000c8fd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:02.435994 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.436168 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364588288}
I20260709 13:11:01.770180 30461 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.774665 30288 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:02.432247 30493 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.797s	user 0.007s	sys 0.003s
I20260709 13:11:01.968730 30490 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.974443 30373 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:02.341677 30500 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:01.973387 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.505944 29081 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:02.594156 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:02.594842 29078 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:02.561519 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.560031 29051 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:02.497475 29080 rpcz_store.cc:269] 0709 13:11:00.657005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.657176 (+   171us) service_pool.cc:224] Handling call
0709 13:11:01.961029 (+1303853us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400146c54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:01.961131 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002534656}
I20260709 13:11:02.560683 29002 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:01.728668 30273 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:02.413340 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:02.096062 30410 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.147634 30337 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.572418 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.377991 30313 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:02.573326 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.507843 29006 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.670017 30395 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:02.441606 30383 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.775s	user 0.009s	sys 0.000s
W20260709 13:11:02.423002 30268 consensus_meta.cc:287] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.000s
I20260709 13:11:02.435412 30482 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.602762 30494 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:02.576555 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.505264 30447 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:02.394789 30439 consensus_meta.cc:287] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.721s	user 0.010s	sys 0.001s
W20260709 13:11:02.404770 30463 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.145s	user 0.011s	sys 0.000s
W20260709 13:11:02.561800 30327 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.011s	sys 0.031s
I20260709 13:11:02.008610 30480 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.434099 30483 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.579896 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.093547 30396 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:02.581050 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:02.561983 30389 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.026s	sys 0.023s
I20260709 13:11:02.206064 30473 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.510205 30496 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:02.665362 30496 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.665953 30496 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:02.517553 30452 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.668385 30452 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:02.671300 30452 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.516269 30484 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:02.340127 30392 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.517400 29151 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:02.596545 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:02.598331 29051 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:02.673192 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.673769 29151 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
W20260709 13:11:02.673826 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 75568.4us
I20260709 13:11:02.674700 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.675685 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:01.733619 30355 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.676360 30355 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:02.108227 30511 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:02.676854 30355 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.676975 30511 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:02.677397 30355 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:02.677588 30511 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.105010 30267 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:02.012221 30356 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:02.595037 29081 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.680974 29081 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:02.510385 30328 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:02.516724 30509 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:02.433769 30369 consensus_meta.cc:287] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.903s	user 0.007s	sys 0.002s
I20260709 13:11:02.607223 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.696884 29156 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:02.612352 30313 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:02.613611 30504 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:02.614121 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.702225 29152 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:02.602111 30348 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:02.618233 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.619138 29006 raft_consensus.cc:2410] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:02.620721 30288 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:02.600140 30283 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:02.599685 30499 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.600204 30490 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:02.625557 30479 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.916s	user 0.011s	sys 0.000s
W20260709 13:11:02.626861 30314 consensus_meta.cc:287] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.918s	user 0.010s	sys 0.001s
I20260709 13:11:02.628470 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1275) took 2162 ms. Trace:
I20260709 13:11:01.736150 30503 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.629850 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.602525 30273 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:02.227682 30432 consensus_meta.cc:287] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.737s	user 0.003s	sys 0.006s
W20260709 13:11:02.642542 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 142875us
I20260709 13:11:02.645074 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.606256 29004 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:11:02.650254 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.640533 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:02.648197 30414 consensus_meta.cc:287] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.000s
I20260709 13:11:02.660207 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:02.662850 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 165835us
I20260709 13:11:02.643085 30494 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.640553 30500 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:02.642802 30202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.007s	sys 0.022s
I20260709 13:11:02.637977 30383 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.786513 30383 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.660830 30480 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:02.664774 30341 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:02.676441 29151 raft_consensus.cc:2410] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:11:02.788493 30362 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:02.788635 30521 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:11:02.788769 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 944) took 2149 ms. Trace:
I20260709 13:11:02.789196 29151 rpcz_store.cc:276] 0709 13:11:00.638935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.787927 (+2148992us) service_pool.cc:224] Handling call
0709 13:11:02.788286 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.788457 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.788483 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:02.788524 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:02.788559 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.788619 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.788716 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:02.790414 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.791316 30503 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.513360 30506 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:02.792034 29154 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:02.792250 30506 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.793558 29154 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:02.794370 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1697) took 2117 ms. Trace:
I20260709 13:11:02.794802 29154 rpcz_store.cc:276] 0709 13:11:00.677392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.792942 (+2115550us) service_pool.cc:224] Handling call
0709 13:11:02.793311 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.794073 (+   762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.794308 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:02.777381 30494 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:02.796414 29001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1294) took 1322 ms (client timeout 1697 ms). Trace:
I20260709 13:11:02.796640 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1699) took 2118 ms. Trace:
W20260709 13:11:02.796811 29001 rpcz_store.cc:269] 0709 13:11:01.473674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.533404 (+ 59730us) service_pool.cc:224] Handling call
0709 13:11:02.796249 (+1262845us) spinlock_profiling.cc:243] Waited 58 ms on lock 0x7b5400311ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:02.796363 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133440896}
I20260709 13:11:02.797013 29154 rpcz_store.cc:276] 0709 13:11:00.677943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.795846 (+2117903us) service_pool.cc:224] Handling call
0709 13:11:02.796157 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.796315 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.796344 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:02.796383 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:02.796431 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.796493 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.796594 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:02.798138 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.681983 29159 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
W20260709 13:11:02.692346 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 184853us
I20260709 13:11:02.762359 30451 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:02.777911 30202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.007s	sys 0.022s
I20260709 13:11:02.801074 30451 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:02.801821 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1704) took 2120 ms. Trace:
I20260709 13:11:02.802204 29159 rpcz_store.cc:276] 0709 13:11:00.681077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.801146 (+2120069us) service_pool.cc:224] Handling call
0709 13:11:02.801387 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.801541 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.801565 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:02.801604 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:02.801637 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.801688 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.801775 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:02.661948 30389 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.027s	sys 0.023s
I20260709 13:11:02.495828 29005 rpcz_store.cc:276] 0709 13:11:00.465930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.471680 (+  5750us) service_pool.cc:224] Handling call
0709 13:11:00.500451 (+ 28771us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:01.596594 (+1096143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.636407 (+ 39813us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:01.703341 (+ 66934us) log.cc:844] Serialized 26 byte log entry
0709 13:11:01.948683 (+245342us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:01.948755 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.948884 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:01.957193 (+  8309us) raft_consensus.cc:1685] finished
0709 13:11:01.957220 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.957390 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14468,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":1224734,"threads_started":2,"wal-append.queue_time_us":230660}
I20260709 13:11:02.773874 30414 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.805598 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.806205 29005 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:02.806916 30414 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:02.807070 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.772825 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1728) took 1001 ms. Trace:
I20260709 13:11:02.807696 29005 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:02.808333 30451 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.808911 29154 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:02.661852 30396 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:02.710958 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.809420 30396 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:02.809621 29006 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:02.810127 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1722) took 1077 ms. Trace:
I20260709 13:11:02.810410 29006 rpcz_store.cc:276] 0709 13:11:01.732524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.710661 (+978137us) service_pool.cc:224] Handling call
0709 13:11:02.810075 (+ 99414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:02.811228 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:02.811725 29006 raft_consensus.cc:2376] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3. Re-sending same reply.
I20260709 13:11:02.808029 29004 rpcz_store.cc:276] 0709 13:11:01.771594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.973161 (+201567us) service_pool.cc:224] Handling call
0709 13:11:02.766335 (+793174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:02.812489 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.813072 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.813860 29004 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:02.813925 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:02.814808 29002 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:02.815193 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1719) took 1085 ms. Trace:
I20260709 13:11:02.815440 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 89) took 2119 ms. Trace:
I20260709 13:11:02.816421 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1700) took 2138 ms. Trace:
W20260709 13:11:02.816969 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.818300 30355 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.820013 30283 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:02.820565 30283 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.821270 30355 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:02.795017 30447 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:02.822614 30283 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:02.813828 29006 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:02.826082 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:02.815938 30396 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:02.827380 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:02.827921 29713 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:02.828284 30328 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.828516 30480 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.828733 30328 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:02.697985 30509 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.719413 29008 rpcz_store.cc:276] 0709 13:11:00.466501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.604927 (+138426us) service_pool.cc:224] Handling call
0709 13:11:00.605325 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:00.605743 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:00.605782 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:00.634053 (+ 28271us) log.cc:844] Serialized 26 byte log entry
0709 13:11:01.001790 (+367737us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:01.530379 (+528589us) log.cc:844] Serialized 14 byte log entry
0709 13:11:01.551134 (+ 20755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.551206 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:02.431572 (+880366us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:02.628122 (+196550us) raft_consensus.cc:1685] finished
0709 13:11:02.628247 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.628410 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12577,"prepare.queue_time_us":658079,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":16470,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":35839,"spinlock_wait_cycles":4948736,"thread_start_us":545802,"threads_started":2,"wal-append.queue_time_us":852364}
I20260709 13:11:02.862746 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.650023 30463 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.734921 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.761890 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.762477 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:02.709194 30313 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:02.765799 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1723) took 1033 ms. Trace:
I20260709 13:11:02.706897 30433 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:02.777117 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 164815us
I20260709 13:11:02.864773 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.864876 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:02.865495 29150 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:02.737561 30475 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.009s	sys 0.032s
I20260709 13:11:02.735277 30314 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:02.763161 30273 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:02.762367 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 158451us
W20260709 13:11:02.876390 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46749.6us
I20260709 13:11:02.803277 30518 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:02.763334 30357 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.700291 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.808804 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.881608 29005 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 1.
I20260709 13:11:02.882916 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:02.647712 30508 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.721333 30499 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:02.884369 30508 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.823760 30447 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.830073 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 86) took 2141 ms. Trace:
I20260709 13:11:02.889786 29006 rpcz_store.cc:276] 0709 13:11:00.689078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.813319 (+2124241us) service_pool.cc:224] Handling call
0709 13:11:02.813556 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.824005 (+ 10449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.830016 (+  6011us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:02.867795 30475 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.018s	sys 0.046s
I20260709 13:11:02.890799 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:02.891359 29006 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.730583 29081 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:02.816753 29154 rpcz_store.cc:276] 0709 13:11:00.678226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.798065 (+2119839us) service_pool.cc:224] Handling call
0709 13:11:02.798336 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.816166 (+ 17830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.816359 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:02.816984 29009 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:02.799317 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:02.815486 29002 rpcz_store.cc:276] 0709 13:11:01.730205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.813688 (+1083483us) service_pool.cc:224] Handling call
0709 13:11:02.815141 (+  1453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:02.803767 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1705) took 2122 ms. Trace:
I20260709 13:11:02.791894 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 946) took 2152 ms. Trace:
W20260709 13:11:02.860591 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 353707us
I20260709 13:11:02.864611 29010 rpcz_store.cc:276] 0709 13:11:01.732786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.649945 (+917159us) service_pool.cc:224] Handling call
0709 13:11:02.765716 (+115771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:02.865388 29008 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
W20260709 13:11:02.868845 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 106437us
I20260709 13:11:02.873823 29713 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:02.800164 30500 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.828441 30396 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:02.732549 30490 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:02.886013 29084 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:02.777393 29007 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:02.732350 30479 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:02.886915 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 207844us
I20260709 13:11:02.867206 30504 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:11:02.892112 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1252) took 1522 ms. Trace:
I20260709 13:11:02.831534 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:02.829082 30480 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:02.992450 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 943) took 2349 ms. Trace:
I20260709 13:11:02.995607 30313 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.084868 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 90) took 2388 ms. Trace:
I20260709 13:11:03.087260 30490 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.084304 29158 rpcz_store.cc:276] 0709 13:11:00.637988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.638115 (+   127us) service_pool.cc:224] Handling call
0709 13:11:00.638473 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.564404 (+925931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.986819 (+1422415us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54002b66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:02.987076 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2097462528}
I20260709 13:11:03.085322 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1721) took 1354 ms. Trace:
I20260709 13:11:02.874812 30433 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.088327 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:03.089784 29158 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:02.923761 30496 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:11:03.090214 30480 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.086047 29003 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
W20260709 13:11:02.973170 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.085233 29151 rpcz_store.cc:276] 0709 13:11:00.639483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.791214 (+2151731us) service_pool.cc:224] Handling call
0709 13:11:02.791458 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.791618 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.791647 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:02.791686 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:02.791714 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.791763 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.791846 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.091368 30509 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.092352 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:03.093504 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.093169 29081 rpcz_store.cc:276] 0709 13:11:01.369586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.673528 (+303942us) service_pool.cc:224] Handling call
0709 13:11:01.673848 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.095867 (+422019us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b540019ada0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157b8dc0 00007f41158431c0 00007f4115844231 00007f4115840d8f 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c
0709 13:11:02.891824 (+795957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.892060 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142492416}
I20260709 13:11:03.093874 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:03.091781 30496 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:03.095067 29157 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:03.095438 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.877336 30451 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:02.888700 30373 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:11:02.815959 29004 rpcz_store.cc:276] 0709 13:11:00.695977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.814679 (+2118702us) service_pool.cc:224] Handling call
0709 13:11:02.814957 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.815118 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.815145 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:02.815191 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:02.815222 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.815277 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.815393 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.097517 30509 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:02.946619 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.087203 29009 rpcz_store.cc:276] 0709 13:11:00.696564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.816450 (+2119886us) service_pool.cc:224] Handling call
0709 13:11:02.816695 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.084600 (+267905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.084814 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.091750 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.877354 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:02.892591 29082 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:03.088703 29007 rpcz_store.cc:276] 0709 13:11:01.730790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.634335 (+903545us) service_pool.cc:224] Handling call
0709 13:11:03.085262 (+450927us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.099016 29150 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
W20260709 13:11:03.099081 29079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1238) took 2429 ms (client timeout 2041 ms). Trace:
I20260709 13:11:03.099841 29125 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:03.100615 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 269131us
W20260709 13:11:02.909487 30397 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
W20260709 13:11:03.102654 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:03.261940 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 384578us
I20260709 13:11:03.107007 29158 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:03.028474 30357 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.096030 30479 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:03.084304 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:03.084360 29159 rpcz_store.cc:276] 0709 13:11:00.681314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.803086 (+2121772us) service_pool.cc:224] Handling call
0709 13:11:02.803334 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.803487 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.803511 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:02.803550 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:02.803578 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.803628 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.803719 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.119992 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:03.272414 29153 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:03.272675 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:03.273315 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:02.883939 29005 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:03.122424 28976 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:03.126945 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:03.274567 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.275275 29006 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:03.127635 28906 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52660 (request call id 245) took 5573 ms. Trace:
W20260709 13:11:03.118804 29048 connection.cc:256] RPC call timeout handler was delayed by -1.2491129129998626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 13:11:03.276501 29006 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.277175 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1289) took 1894 ms. Trace:
W20260709 13:11:03.277685 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.042444 30511 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.175225 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1287) took 1806 ms. Trace:
W20260709 13:11:03.195665 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.281821 29008 rpcz_store.cc:276] 0709 13:11:01.369103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.174134 (+1805031us) service_pool.cc:224] Handling call
0709 13:11:03.174541 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.174766 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.174809 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.174882 (+    73us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:03.174932 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.175009 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.175152 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.207736 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:03.118341 30386 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:03.217612 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 300265us
I20260709 13:11:03.130509 30496 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:03.259043 29151 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:03.259956 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:03.196290 29711 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.271365 29154 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:03.265379 29051 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:03.259989 29079 rpcz_store.cc:269] 0709 13:11:00.670029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.212059 (+542030us) service_pool.cc:224] Handling call
0709 13:11:03.098865 (+1886806us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540034e254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.099026 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415838464}
I20260709 13:11:03.267561 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1288) took 1893 ms. Trace:
I20260709 13:11:03.044430 30506 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.172665 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 92) took 1807 ms. Trace:
I20260709 13:11:03.213131 29076 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:03.345745 29156 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
W20260709 13:11:03.274598 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 152330us
I20260709 13:11:03.277498 29006 rpcz_store.cc:276] 0709 13:11:01.382691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.275937 (+1893246us) service_pool.cc:224] Handling call
0709 13:11:03.276335 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.276950 (+   615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.277127 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:03.282146 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 182364us
W20260709 13:11:03.284694 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 77037.1us
I20260709 13:11:03.285449 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:03.286290 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:03.203697 30451 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.336376 30530 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.284593 28975 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:03.368983 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 84466.6us
I20260709 13:11:03.333901 30518 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.337620 29150 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.373723 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 949) took 2659 ms. Trace:
I20260709 13:11:03.373981 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:03.034670 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 138667us
I20260709 13:11:03.290102 29008 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:03.338830 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 89) took 2644 ms. Trace:
I20260709 13:11:03.384872 29151 rpcz_store.cc:276] 0709 13:11:00.694010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.338042 (+2644032us) service_pool.cc:224] Handling call
0709 13:11:03.338323 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.338510 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.338543 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.338604 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:03.338634 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.338686 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.338783 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:03.295543 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.342453 29009 rpcz_store.cc:276] 0709 13:11:01.374269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.266837 (+1892568us) service_pool.cc:224] Handling call
0709 13:11:03.267123 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.267272 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.267295 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.267332 (+    37us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:03.267359 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.267408 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.267513 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.344027 29010 rpcz_store.cc:276] 0709 13:11:01.365195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.171303 (+1806108us) service_pool.cc:224] Handling call
0709 13:11:03.171795 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.172129 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.172176 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.172243 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:03.172325 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.172404 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.172582 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.347414 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:03.361385 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:03.353351 30202 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:03.231612 30355 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:03.359476 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:03.336665 30496 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.337845 30389 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.333690 30490 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.280964 30463 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.281306 30527 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:03.374797 29153 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:03.363938 29682 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:03.368271 30451 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:03.289579 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:03.418851 29158 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.374125 29150 rpcz_store.cc:276] 0709 13:11:00.714054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.337125 (+2623071us) service_pool.cc:224] Handling call
0709 13:11:03.337441 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.373477 (+ 36036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.373667 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:03.347604 30533 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:11:03.421819 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 57962.2us
I20260709 13:11:03.279363 30480 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
I20260709 13:11:03.271055 30509 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:03.393169 30202 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:03.399153 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.278456 30499 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.374023 30447 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:03.335693 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:03.278661 30357 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:03.412567 30535 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 13:11:03.422552 29682 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:03.414754 29709 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:03.427397 30202 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.381918 30386 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.429761 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1715) took 1991 ms. Trace:
I20260709 13:11:03.446259 29150 rpcz_store.cc:276] 0709 13:11:01.438550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.428789 (+1990239us) service_pool.cc:224] Handling call
0709 13:11:03.429159 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.429367 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.429403 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.429460 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:03.429501 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.429567 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.429693 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.414589 30496 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:03.391492 30494 consensus_meta.cc:287] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.591s	user 0.008s	sys 0.004s
I20260709 13:11:03.419356 30527 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:03.412391 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:03.434826 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.343372 30414 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 13:11:03.418110 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56802.5us
I20260709 13:11:03.423210 30389 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.414935 30355 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.466055 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.433050 30383 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:03.428642 30525 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:03.468768 30414 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:03.469275 30479 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:03.469529 30506 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.423038 30490 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.436177 30447 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:03.455663 30527 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.489181 30525 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.491422 30383 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:03.433244 30509 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:03.443804 30313 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:11:03.477249 30540 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:11:03.464682 30539 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:11:03.470072 30504 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:03.457823 30530 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:03.487444 30414 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:03.487630 30479 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:03.497576 30447 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.514367 30357 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.516541 30510 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:03.521667 30509 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.545403 30479 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.527118 30511 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:03.536998 30373 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:03.553483 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 217870us
I20260709 13:11:03.542073 30313 raft_consensus.cc:697] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:03.510104 29683 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:03.549459 30510 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:03.552538 30357 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:03.551195 30530 raft_consensus.cc:697] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:03.571214 30451 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.573101 30357 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:03.593351 30490 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.599308 30433 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.593422 30530 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.606309 30395 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.611253 30500 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.615000 30479 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.607690 30508 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.617543 30496 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.628916 30496 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:03.633867 30527 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.658012 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:03.370445 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.661444 29001 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:03.661224 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 290661us
I20260709 13:11:03.411033 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:03.662885 28975 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.341020 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:03.396752 29006 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:03.384443 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:03.664273 29007 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:03.665149 29005 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.390518 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1291) took 1985 ms. Trace:
I20260709 13:11:03.667147 29006 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:03.667238 29008 rpcz_store.cc:276] 0709 13:11:01.404818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.282987 (+1878169us) service_pool.cc:224] Handling call
0709 13:11:03.283258 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.389589 (+106331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.390441 (+   852us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.668678 29004 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.669602 29008 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.670264 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1711) took 1944 ms. Trace:
I20260709 13:11:03.670583 29008 rpcz_store.cc:276] 0709 13:11:01.726055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.668778 (+1942723us) service_pool.cc:224] Handling call
0709 13:11:03.669295 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.670028 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.670196 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.672505 29008 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:11:03.673281 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1714) took 1944 ms. Trace:
I20260709 13:11:03.673873 28975 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:03.676301 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 196085us
W20260709 13:11:03.677650 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.677958 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1717) took 2136 ms. Trace:
I20260709 13:11:03.463366 29158 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:03.715596 29153 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.717008 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 94) took 2054 ms. Trace:
I20260709 13:11:03.717424 29158 rpcz_store.cc:276] 0709 13:11:01.662203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.715744 (+2053541us) service_pool.cc:224] Handling call
0709 13:11:03.716178 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.716570 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.716594 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.716653 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:03.716695 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.716743 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.716839 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.705746 29154 rpcz_store.cc:276] 0709 13:11:01.541116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.508674 (+1967558us) service_pool.cc:224] Handling call
0709 13:11:03.676580 (+167906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.677170 (+   590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.677206 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.677282 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:03.677353 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.677452 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.677597 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.721213 29154 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:03.721508 30527 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.722143 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1730) took 1108 ms. Trace:
I20260709 13:11:03.722666 29154 rpcz_store.cc:276] 0709 13:11:02.613744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.720013 (+1106269us) service_pool.cc:224] Handling call
0709 13:11:03.720592 (+   579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.721770 (+  1178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.722054 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:03.679085 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 413625us
I20260709 13:11:03.725419 30396 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.725987 29009 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
W20260709 13:11:03.726336 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.726140 30396 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.726749 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1292) took 2313 ms. Trace:
W20260709 13:11:03.727183 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.727125 29009 rpcz_store.cc:276] 0709 13:11:01.413468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.398177 (+1984709us) service_pool.cc:224] Handling call
0709 13:11:03.725145 (+326968us) spinlock_profiling.cc:243] Waited 899 us on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:03.725562 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.726492 (+   930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.726687 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067328}
I20260709 13:11:03.684929 29151 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:03.689105 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:03.736730 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 47992us
I20260709 13:11:03.626542 30509 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.692807 30355 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.740603 30355 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:03.740782 30396 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.741034 30355 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:03.741470 30509 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:03.586686 30357 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:03.510802 30328 consensus_meta.cc:287] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.000s
I20260709 13:11:03.744693 30328 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.746162 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1709) took 2020 ms. Trace:
I20260709 13:11:03.746611 29007 rpcz_store.cc:276] 0709 13:11:01.725591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.665529 (+1939938us) service_pool.cc:224] Handling call
0709 13:11:03.745107 (+ 79578us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:03.745401 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.745731 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.745758 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.745822 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:03.745862 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.745928 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.746077 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91197696}
I20260709 13:11:03.695245 30389 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.496546 30426 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
W20260709 13:11:03.750823 29084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 118) took 2190 ms (client timeout 2025 ms). Trace:
I20260709 13:11:03.751134 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:03.751298 29084 rpcz_store.cc:269] 0709 13:11:01.560437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.560842 (+   405us) service_pool.cc:224] Handling call
0709 13:11:03.750293 (+2189451us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54000c8ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.750591 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817828736}
I20260709 13:11:03.752149 29009 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:03.753194 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:03.753448 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 80379.4us
I20260709 13:11:03.754225 30510 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.755362 30395 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:03.755857 30395 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.756321 30395 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:03.756592 29152 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:03.757836 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1725) took 1318 ms. Trace:
I20260709 13:11:03.758173 29152 rpcz_store.cc:276] 0709 13:11:02.439630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.708073 (+268443us) service_pool.cc:224] Handling call
0709 13:11:03.757597 (+1049524us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54003ed0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.757766 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2051417856}
I20260709 13:11:03.758924 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:03.759783 29007 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:03.761093 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 116) took 2286 ms. Trace:
I20260709 13:11:03.761597 29082 rpcz_store.cc:276] 0709 13:11:01.474511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.712731 (+238220us) service_pool.cc:224] Handling call
0709 13:11:01.713018 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.258664 (+1545646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.760501 (+501837us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400146754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:03.760837 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2706892544}
I20260709 13:11:03.692611 30313 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.563313 30383 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.766029 30395 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.476769 30386 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:03.696983 29003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1727) took 1925 ms (client timeout 2169 ms). Trace:
W20260709 13:11:03.767345 29003 rpcz_store.cc:269] 0709 13:11:01.771272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.659924 (+888652us) service_pool.cc:224] Handling call
0709 13:11:03.696626 (+1036702us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54001856d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.696789 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719653248}
I20260709 13:11:03.755395 28975 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.768671 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:03.545327 30414 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.770290 30527 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:03.697580 30391 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:03.696439 30356 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.701304 30479 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:03.702245 30480 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.703497 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1701) took 3023 ms. Trace:
I20260709 13:11:03.778242 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:03.511250 30525 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.275954 28906 rpcz_store.cc:276] 0709 13:10:57.554108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.554612 (+   504us) service_pool.cc:224] Handling call
0709 13:11:03.127543 (+5572931us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:00.539974 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:00.590590 (+ 50616us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:00.590613 (+    23us) write_op.cc:435] Acquired schema lock
0709 13:11:00.590624 (+    11us) tablet.cc:662] Decoding operations
0709 13:11:00.590950 (+   326us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:00.590985 (+    35us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:00.591001 (+    16us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:00.591128 (+   127us) tablet.cc:701] Row locks acquired
0709 13:11:00.591138 (+    10us) write_op.cc:260] PREPARE: finished
0709 13:11:00.591234 (+    96us) write_op.cc:270] Start()
0709 13:11:00.591314 (+    80us) write_op.cc:276] Timestamp: P: 1783602660591202 usec, L: 0
0709 13:11:00.591336 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:11:00.591838 (+   502us) log.cc:844] Serialized 533 byte log entry
0709 13:11:01.580314 (+988476us) op_driver.cc:464] REPLICATION: finished
0709 13:11:01.801387 (+221073us) write_op.cc:301] APPLY: starting
0709 13:11:01.806607 (+  5220us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:01.806732 (+   125us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:01.806890 (+   158us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:01.844565 (+ 37675us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:01.844779 (+   214us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:11:01.844845 (+    66us) write_op.cc:312] APPLY: finished
0709 13:11:01.876398 (+ 31553us) log.cc:844] Serialized 23 byte log entry
0709 13:11:02.593172 (+716774us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:02.602330 (+  9158us) write_op.cc:454] Released schema lock
0709 13:11:02.606766 (+  4436us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:02.646040 (+ 39274us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:02.646086 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19000,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":201074,"apply.run_cpu_time_us":37881,"apply.run_wall_time_us":837692,"num_ops":1,"prepare.queue_time_us":228554,"prepare.run_cpu_time_us":4838,"prepare.run_wall_time_us":95210,"raft.queue_time_us":846562,"raft.run_cpu_time_us":5694,"raft.run_wall_time_us":154062,"replication_time_us":950190,"spinlock_wait_cycles":3840,"thread_start_us":857126,"threads_started":5,"wal-append.queue_time_us":749830,"wal-append.run_cpu_time_us":22254,"wal-append.run_wall_time_us":1166471}]]}
I20260709 13:11:03.673650 29008 rpcz_store.cc:276] 0709 13:11:01.728530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.671294 (+1942764us) service_pool.cc:224] Handling call
0709 13:11:03.671919 (+   625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:03.672922 (+  1003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.673220 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.707454 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1707) took 1981 ms. Trace:
I20260709 13:11:03.782105 29006 rpcz_store.cc:276] 0709 13:11:01.724119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.665546 (+1941427us) service_pool.cc:224] Handling call
0709 13:11:03.666375 (+   829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.667770 (+  1395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.668027 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.711998 30508 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:03.713151 30447 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:03.678447 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 207397us
I20260709 13:11:03.728550 30550 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:03.729677 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 853) took 3068 ms. Trace:
I20260709 13:11:03.792294 29076 rpcz_store.cc:276] 0709 13:11:00.666719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.556503 (+889784us) service_pool.cc:224] Handling call
0709 13:11:03.118572 (+1562069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.369835 (+251263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.729165 (+359330us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540034e250. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.729444 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3494585600}
I20260709 13:11:03.794054 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1290) took 2398 ms. Trace:
I20260709 13:11:03.794627 29002 rpcz_store.cc:276] 0709 13:11:01.395479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.792597 (+2397118us) service_pool.cc:224] Handling call
0709 13:11:03.793111 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.793490 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.793523 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.793594 (+    71us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:03.793633 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.793703 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.793853 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.797197 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1709) took 3067 ms. Trace:
I20260709 13:11:03.729770 29154 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:03.682621 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 180849us
I20260709 13:11:03.730388 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 93) took 2068 ms. Trace:
I20260709 13:11:03.800318 29159 rpcz_store.cc:276] 0709 13:11:01.661853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.714505 (+2052652us) service_pool.cc:224] Handling call
0709 13:11:03.715286 (+   781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.715746 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.715775 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.715816 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:03.715890 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.715942 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.730185 (+ 14243us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:03.730847 29157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 950) took 3016 ms (client timeout 1824 ms). Trace:
I20260709 13:11:03.731359 29680 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:03.732019 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:03.732402 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 959) took 1756 ms. Trace:
I20260709 13:11:03.735291 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1297) took 2196 ms. Trace:
I20260709 13:11:03.741562 30355 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:03.743643 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 396167us
I20260709 13:11:03.803248 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:03.805545 28976 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:03.769065 30396 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:03.807142 30396 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.807505 30396 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.810456 30355 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.697829 30451 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.772344 30391 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.812577 30391 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.812721 30355 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.776376 29155 rpcz_store.cc:276] 0709 13:11:00.680369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.798894 (+2118525us) service_pool.cc:224] Handling call
0709 13:11:02.799223 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:02.799662 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.799692 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:02.800306 (+   614us) log.cc:844] Serialized 52 byte log entry
0709 13:11:02.876278 (+ 75972us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:03.052739 (+176461us) log.cc:844] Serialized 14 byte log entry
0709 13:11:03.098608 (+ 45869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.702493 (+603885us) spinlock_profiling.cc:243] Waited 84.2 ms on lock 0x7b540010b6a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157b8dc0 00007f4115844a02 00007f4115841dde 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:03.702701 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:03.702797 (+    96us) raft_consensus.cc:1685] finished
0709 13:11:03.702972 (+   175us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.703261 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94714,"prepare.queue_time_us":166154,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":1603,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":196,"spinlock_wait_cycles":193655552,"thread_start_us":75570,"threads_started":1,"wal-append.queue_time_us":87728}
I20260709 13:11:03.465055 30494 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.784884 30395 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:03.691630 30273 consensus_meta.cc:287] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.801s	user 0.004s	sys 0.012s
I20260709 13:11:03.806586 30433 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:03.797745 29151 rpcz_store.cc:276] 0709 13:11:00.730962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.385949 (+2654987us) service_pool.cc:224] Handling call
0709 13:11:03.386278 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:03.735990 (+349712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.796796 (+ 60806us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b5400139f54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:03.797099 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685129472}
W20260709 13:11:03.801440 29157 rpcz_store.cc:269] 0709 13:11:00.714309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.166423 (+452114us) service_pool.cc:224] Handling call
0709 13:11:03.730370 (+2563947us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b540013cc54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.730610 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4011675008}
I20260709 13:11:03.785669 30508 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.802130 29010 rpcz_store.cc:276] 0709 13:11:01.538747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.387327 (+1848580us) service_pool.cc:224] Handling call
0709 13:11:03.700014 (+312687us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:03.711518 (+ 11504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.734759 (+ 23241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.734797 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.734861 (+    64us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:03.734888 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.734948 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.735109 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5095296}
I20260709 13:11:03.767352 29004 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:03.822417 29003 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:03.823015 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.801733 29156 rpcz_store.cc:276] 0709 13:11:01.976259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.697594 (+721335us) service_pool.cc:224] Handling call
0709 13:11:03.723968 (+1026374us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54002a44d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.732220 (+  8252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073081728}
W20260709 13:11:03.833634 30553 log.cc:927] Time spent T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.163s	user 0.001s	sys 0.001s
I20260709 13:11:03.835662 29007 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.832206 30273 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.776768 30357 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.829787 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1732) took 1011 ms. Trace:
I20260709 13:11:03.828905 30508 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:03.832296 28975 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.832396 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:03.828159 30433 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:03.836015 28907 catalog_manager.cc:5697] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:03.877552 28907 catalog_manager.cc:5697] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:03.836531 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1718) took 2106 ms. Trace:
I20260709 13:11:03.851120 29008 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.884323 29007 rpcz_store.cc:276] 0709 13:11:01.729926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.828674 (+2098748us) service_pool.cc:224] Handling call
0709 13:11:03.835405 (+  6731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.836166 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.836437 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.886327 29007 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:03.840848 29712 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.851997 30328 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:03.857255 30426 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:11:03.888206 29712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 435) took 1063 ms. Trace:
I20260709 13:11:03.888693 29712 rpcz_store.cc:276] 0709 13:11:02.824898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.827495 (+  2597us) service_pool.cc:224] Handling call
0709 13:11:03.840247 (+1012752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.887062 (+ 46815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.887891 (+   829us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400650ed0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.888100 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328040320}
I20260709 13:11:03.859644 30414 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.841125 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1712) took 2403 ms. Trace:
I20260709 13:11:03.890518 29153 rpcz_store.cc:276] 0709 13:11:01.437865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.426251 (+1988386us) service_pool.cc:224] Handling call
0709 13:11:03.714867 (+288616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.716080 (+  1213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.840872 (+124792us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540013d150. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.841045 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647736576}
W20260709 13:11:03.837246 29709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49414 (request call id 257) took 2498 ms (client timeout 1951 ms). Trace:
I20260709 13:11:03.883987 30548 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.847564 30335 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:03.872035 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:03.847901 30447 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.848412 30510 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.869756 30267 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:11:03.871994 30525 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.848819 29154 rpcz_store.cc:276] 0709 13:11:02.815753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.728563 (+912810us) service_pool.cc:224] Handling call
0709 13:11:03.729153 (+   590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.800089 (+ 70936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.821951 (+ 21862us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:03.881888 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 150236us
I20260709 13:11:03.901854 30525 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:03.882206 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 320707us
I20260709 13:11:03.882372 30480 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.872386 30433 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.908250 30426 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.859050 29010 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:03.909139 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:03.910625 30383 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:03.911055 30383 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.911407 30383 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.911267 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:03.882591 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:03.873893 30273 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:03.916244 30547 raft_consensus.cc:2764] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:11:03.849907 30357 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.886425 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1717) took 2156 ms. Trace:
I20260709 13:11:03.847188 29004 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
W20260709 13:11:03.868932 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63468.8us
I20260709 13:11:03.891875 30414 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.865325 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.866797 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1716) took 2328 ms. Trace:
I20260709 13:11:03.866036 30451 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:03.891793 29709 rpcz_store.cc:269] 0709 13:11:01.338908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.348266 (+  9358us) service_pool.cc:224] Handling call
0709 13:11:02.413114 (+1064848us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:03.836705 (+1423591us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b5400650ed4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.836984 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776623744}
W20260709 13:11:03.865814 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 64089.6us
I20260709 13:11:03.850144 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1307) took 1017 ms. Trace:
I20260709 13:11:03.933537 29003 rpcz_store.cc:276] 0709 13:11:02.833187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.848987 (+1015800us) service_pool.cc:224] Handling call
0709 13:11:03.849398 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.849758 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.849792 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:03.849829 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:03.849878 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.849929 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.850078 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:03.874749 29005 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:03.884112 30494 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:03.898572 30335 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:03.909194 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 91066.7us
I20260709 13:11:03.920929 29008 rpcz_store.cc:276] 0709 13:11:01.729649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.818956 (+2089307us) service_pool.cc:224] Handling call
0709 13:11:03.850898 (+ 31942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.884944 (+ 34046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.886227 (+  1283us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b540032cdd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.886372 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11554048}
W20260709 13:11:03.935935 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 66191.7us
I20260709 13:11:03.937378 29001 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.943781 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.943964 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1715) took 2215 ms. Trace:
I20260709 13:11:03.944300 29001 rpcz_store.cc:276] 0709 13:11:01.728849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.693923 (+1965074us) service_pool.cc:224] Handling call
0709 13:11:03.937202 (+243279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.942007 (+  4805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.943763 (+  1756us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54003f20d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:03.943889 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555456384}
I20260709 13:11:03.944808 28975 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:03.946372 29005 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:03.946583 30433 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.947798 29001 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:03.948640 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 83516.9us
I20260709 13:11:03.928956 29150 rpcz_store.cc:276] 0709 13:11:01.538336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.453493 (+1915157us) service_pool.cc:224] Handling call
0709 13:11:03.453823 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:03.454121 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.454146 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:03.458945 (+  4799us) log.cc:844] Serialized 26 byte log entry
0709 13:11:03.841972 (+383027us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:03.842055 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.842177 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:03.866384 (+ 24207us) raft_consensus.cc:1685] finished
0709 13:11:03.866411 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.866604 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":500,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":20539,"thread_start_us":332402,"threads_started":1,"wal-append.queue_time_us":356082}
I20260709 13:11:03.932936 30451 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.950994 30536 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:03.942230 30335 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:03.955258 30508 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.951205 30357 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:03.964180 30357 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:03.964310 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 53292.3us
I20260709 13:11:03.964576 30357 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:03.965512 30508 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.966218 30273 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:03.968835 30414 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:03.969393 30414 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.973182 30433 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.971654 30414 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.977232 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:03.977710 29683 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:03.939392 28907 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 3, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:03.978873 30504 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:03.979326 30504 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:03.979839 30504 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.980475 29709 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:03.982066 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:03.982247 30548 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.983647 29711 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:03.985961 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:03.986850 30391 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:03.988217 29009 raft_consensus.cc:2393] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:03.989210 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:03.989045 30383 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:03.990127 30391 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.990271 29680 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:03.979499 29683 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:03.979517 28907 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 3, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:03.951747 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1306) took 1117 ms. Trace:
I20260709 13:11:03.954000 30447 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:03.953192 30451 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:03.951627 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.013960 29002 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:04.014776 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:04.016866 29712 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:04.017318 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1730) took 2233 ms. Trace:
I20260709 13:11:04.017683 29002 rpcz_store.cc:276] 0709 13:11:01.783762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.843831 (+2060069us) service_pool.cc:224] Handling call
0709 13:11:04.008615 (+164784us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:04.014405 (+  5790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:04.017118 (+  2713us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400228650. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:04.017264 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377738112}
I20260709 13:11:04.019469 29002 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:04.020089 30548 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:04.020243 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 66045us
I20260709 13:11:04.020999 30548 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:04.028596 29076 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:04.007452 29004 rpcz_store.cc:276] 0709 13:11:02.832973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.846355 (+1013382us) service_pool.cc:224] Handling call
0709 13:11:03.846839 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.936413 (+ 89574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.950109 (+ 13696us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:04.030501 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.031339 29084 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:04.032172 30494 consensus_queue.cc:1059] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:04.032656 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.033423 29082 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:04.020141 30509 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:04.034668 30509 raft_consensus.cc:697] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:04.035948 30509 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.036994 30383 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:03.990736 30273 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:04.015337 28907 catalog_manager.cc:5697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 3, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:04.045768 30500 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:11:04.025180 29007 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:04.048456 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.047477 30504 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.049804 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.050475 29153 raft_consensus.cc:2376] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3. Re-sending same reply.
I20260709 13:11:04.051007 30357 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.051502 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.052287 30504 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.052809 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.053339 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.053653 29157 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:04.053996 29082 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:04.054636 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.025404 30391 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.051504 29717 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.026898 30447 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:04.003224 30396 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.056572 30391 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.064795 30273 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.066849 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1727) took 1626 ms. Trace:
I20260709 13:11:04.045782 30386 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:11:04.068738 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:04.068594 30397 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:04.069348 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 70456.8us
I20260709 13:11:04.058295 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.068807 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:04.058410 30447 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.075691 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:04.076195 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 68407.4us
I20260709 13:11:04.076718 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.076962 29125 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:04.076979 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 57255.4us
I20260709 13:11:04.077117 30355 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.073729 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.078116 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.078576 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.081065 29156 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:04.086489 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.086822 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.086597 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.087982 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.087929 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.067262 29158 rpcz_store.cc:276] 0709 13:11:02.440158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.718523 (+1278365us) service_pool.cc:224] Handling call
0709 13:11:03.719070 (+   547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.752509 (+ 33439us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:03.829218 (+ 76709us) log.cc:844] Serialized 14 byte log entry
0709 13:11:04.066308 (+237090us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b1800287200. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:11:04.066511 (+   203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:04.066563 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:04.066601 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:04.066667 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:04.066788 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18979,"prepare.queue_time_us":107733,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":63391,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":4476288,"thread_start_us":119859,"threads_started":2,"wal-append.queue_time_us":77135}
I20260709 13:11:04.085652 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.091822 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.093338 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.095535 30391 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.087934 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:04.072295 30386 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:11:04.100389 29079 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:04.103363 29717 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:04.104470 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.104293 30328 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:04.104733 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:04.105189 29081 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:04.105497 30545 raft_consensus.cc:3055] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:04.105855 30545 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:04.107015 30548 consensus_queue.cc:1059] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:04.106807 30545 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.096431 30355 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:11:04.102388 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.111979 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:04.107873 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.114846 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.115471 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.117909 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.127394 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.128129 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.128159 29080 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:04.129937 30389 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:04.130662 30505 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:04.112511 30530 raft_consensus.cc:3055] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:04.139734 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.145175 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.112907 30525 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:04.136940 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:04.124212 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.155844 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.156306 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.156258 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.165192 29079 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.118029 29711 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
W20260709 13:11:04.173110 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 66245.8us
I20260709 13:11:04.173933 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.164844 30549 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:11:04.175387 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:04.175621 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.178334 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.117048 30494 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:11:04.180924 30479 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:04.155951 30550 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:04.147521 30530 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:04.187934 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:04.191437 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 108873us
I20260709 13:11:04.192535 29158 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.138751 30389 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:11:04.170238 30357 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.179373 30525 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:04.200439 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.188285 30550 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:04.204258 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.209501 29152 raft_consensus.cc:2393] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:04.177834 30510 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:11:04.179998 30451 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:04.213606 30561 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:11:04.214839 30568 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.215256 30451 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.199709 30545 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.202154 30391 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:04.213255 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:04.220491 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:04.225028 30510 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:04.226501 30510 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:04.226537 30496 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 13:11:04.227550 30510 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.228842 30510 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.210809 30396 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.232627 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.239708 29001 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.224421 30391 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:04.244952 30560 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:11:04.245047 29078 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:04.246024 30566 raft_consensus.cc:3055] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:04.245762 30391 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.246387 30566 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:04.247547 30202 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:11:04.247845 30557 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:11:04.248728 30573 consensus_queue.cc:1059] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:04.248632 30530 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:04.241389 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 53091.6us
I20260709 13:11:04.205799 30335 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:04.271057 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 81565us
I20260709 13:11:04.273383 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1304) took 1441 ms. Trace:
I20260709 13:11:04.247380 30566 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.279747 30530 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.279872 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.278291 28975 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:04.215360 30568 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.299896 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.278978 30395 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:04.310808 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 99296.6us
W20260709 13:11:04.311555 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 49064.2us
I20260709 13:11:04.278744 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:04.279299 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:04.273909 29006 rpcz_store.cc:276] 0709 13:11:02.832453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.844621 (+1012168us) service_pool.cc:224] Handling call
0709 13:11:03.845098 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.845428 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:04.009424 (+163996us) log.cc:844] Serialized 14 byte log entry
0709 13:11:04.273023 (+263599us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:04.273078 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:04.273120 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:04.273192 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:04.273325 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":158912,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":336,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":166,"thread_start_us":367071,"threads_started":2,"wal-append.queue_time_us":229852}
I20260709 13:11:04.249428 30525 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.221932 30550 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.309221 30355 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.330742 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:04.335129 30564 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:11:04.335709 29005 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:04.335713 30451 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:04.336911 30451 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.337313 30451 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.338827 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.339128 29158 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:04.339792 29076 raft_consensus.cc:2410] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:11:04.340708 30562 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:11:04.348851 30544 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:04.349434 30544 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:04.350494 30544 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.352260 30544 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.352638 30566 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.357510 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.361263 29154 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:04.365629 30568 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:04.367223 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.368088 29153 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:04.371409 30426 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:11:04.373790 30576 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:11:04.355525 30451 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.375821 30395 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:04.376331 30395 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.379942 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.380458 30355 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:04.382330 30355 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:04.382704 30355 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:04.383698 30355 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.371950 30426 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:04.384763 30355 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.385387 30426 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.387051 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:04.387143 30426 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.392566 30328 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:04.393183 30328 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:04.328724 29007 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:04.398000 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.336663 30567 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
W20260709 13:11:04.392400 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 68272.3us
I20260709 13:11:04.400292 29079 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
W20260709 13:11:04.451009 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:04.454674 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:04.455376 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.456122 29051 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:04.461714 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.454186 30573 consensus_queue.cc:1243] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer ca45420e033847489ad989c1c6eed3ef log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:04.465255 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.449493 30451 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.475960 29080 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:04.480394 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.483563 29717 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:04.490379 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:04.474792 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 94034.2us
I20260709 13:11:04.478688 30335 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:04.505789 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.483388 30536 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:04.494995 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.517465 29079 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:04.527611 29081 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:04.529500 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.585980 29078 raft_consensus.cc:2410] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:11:04.532148 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.598968 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:04.532647 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.540017 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:04.540937 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.541489 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.541913 30536 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.543036 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:04.544690 30386 raft_consensus.cc:3055] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:04.545650 30582 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.546751 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:04.517088 30567 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:11:04.518908 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.555416 29711 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:04.543769 30565 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:04.559288 30590 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:04.566212 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:04.569013 30395 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.570086 30267 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.572671 30574 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:04.574412 30328 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.504798 30568 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:04.626530 29002 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:04.633343 29003 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:04.647075 29680 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:04.648698 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.664209 29001 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:04.663748 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:04.698318 29006 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:04.698438 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.616362 30386 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:04.702545 29009 raft_consensus.cc:2393] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:04.706257 29085 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:04.707084 29079 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:04.621296 29077 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:04.647687 30494 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:11:04.707957 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.632251 30565 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:04.714684 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 92810.5us
I20260709 13:11:04.707867 29708 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 2: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:11:04.717595 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 392320us
I20260709 13:11:04.718073 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.718942 29150 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.720679 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.721706 29151 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:04.723976 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.724565 29007 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:04.724861 29155 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:04.725335 30328 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.728665 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:04.716506 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 260408us
W20260709 13:11:04.733073 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 86057.1us
I20260709 13:11:04.734123 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.734719 29157 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.734361 30386 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.735805 30386 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.736397 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:04.737251 29156 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:04.732365 29051 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:04.716076 30267 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.740692 30577 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:04.741123 30577 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.741242 29079 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:04.741521 30577 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:04.740473 30591 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:11:04.744228 29153 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:04.744203 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 124791us
I20260709 13:11:04.731148 30530 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:04.645661 30590 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.705606 30568 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:04.716593 30494 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:04.742165 30504 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:04.772179 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:04.772269 29122 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:04.736001 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 169863us
W20260709 13:11:04.782524 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:04.783037 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:04.793073 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.793496 29076 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:04.770896 30267 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:04.787564 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.794618 30267 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:11:04.795573 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.766170 30590 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.797488 29156 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:04.806103 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.779942 30383 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:04.811800 29085 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:04.813516 30391 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:04.815990 29154 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:04.816175 30594 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:04.816491 29153 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:04.818733 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:04.824391 29150 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:04.823640 29157 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:04.820166 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:04.823969 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.860518 29077 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:04.864306 30494 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.867075 29156 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:04.869240 30568 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.871500 29079 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:11:04.873956 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:04.876530 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 79586.8us
W20260709 13:11:04.878327 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 106215us
I20260709 13:11:04.879237 29680 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:04.880671 30594 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:04.865401 30494 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:04.879283 30395 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.892866 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.895956 29712 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:04.709864 30574 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:04.900493 29084 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:04.901943 29003 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:04.905673 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.896760 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.897934 30397 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:11:04.921720 29708 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:04.924702 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:04.926736 30588 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:11:04.930768 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:04.934270 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.872761 30565 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:04.945513 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:04.947718 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.932559 30536 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:04.932801 30383 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:04.949947 29713 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:04.953997 30604 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:04.955363 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:04.966287 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:04.941783 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 138268us
I20260709 13:11:04.969043 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.039968 29085 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:05.043725 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.044703 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:05.045380 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 104590us
I20260709 13:11:04.947217 30573 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.092s
I20260709 13:11:05.047673 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:04.952936 29155 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:05.051723 29713 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:05.061903 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.061817 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.064113 29085 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.062644 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.055325 29683 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:05.115464 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:05.118871 30597 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:11:05.129074 30550 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:05.139954 30584 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.148203 30566 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:05.144946 30386 consensus_queue.cc:1059] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:11:05.140108 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.077139 30594 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.118396 30480 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.157109 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.147718 29711 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:11:05.170352 29079 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:11:05.126428 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 81997.6us
I20260709 13:11:05.180899 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.176208 30479 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:05.147148 30577 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:05.188797 29122 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:05.196226 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.193462 30328 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:05.201695 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.201615 30585 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:05.185119 30479 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:05.205256 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.205350 29006 raft_consensus.cc:2368] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4.
I20260709 13:11:05.206655 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.181701 30590 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.201871 29715 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:05.212591 30590 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:05.209836 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:05.210055 29048 leader_election.cc:336] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.523s (SENT)
I20260709 13:11:05.213503 29081 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.213541 29048 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:05.237555 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.245847 30574 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:05.253572 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.254204 30577 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:05.254545 29006 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.254068 30479 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.255357 30479 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:05.258071 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 48077.4us
W20260709 13:11:05.259219 29048 leader_election.cc:336] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.550s (SENT)
I20260709 13:11:05.259593 29048 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.257356 30577 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.259641 29079 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:05.255089 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.272197 30596 raft_consensus.cc:3055] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:05.272647 29004 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.273348 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.267788 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.274742 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.275107 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.275383 29083 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:05.276247 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.276618 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:05.278210 29077 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
W20260709 13:11:05.278918 30582 consensus_meta.cc:287] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.613s	user 0.002s	sys 0.005s
I20260709 13:11:05.279119 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.279312 30582 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.267534 29085 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:05.280253 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.281697 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.280859 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.284523 29085 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:05.272689 30596 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:05.286077 30596 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.288028 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.289580 29079 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:05.290654 29048 leader_election.cc:336] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.604s (SENT)
I20260709 13:11:05.291203 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:05.304325 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 88760.1us
I20260709 13:11:05.306074 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:05.306301 29081 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:05.306653 29085 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:05.307253 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.307624 30582 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.287263 30596 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:05.286091 30603 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.353s	user 0.002s	sys 0.002s
I20260709 13:11:05.281739 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.312860 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:05.324343 29048 leader_election.cc:336] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.614s (SENT)
I20260709 13:11:05.324982 29048 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.307550 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.320853 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:05.318192 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 117379us
I20260709 13:11:05.320869 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:05.328430 29006 raft_consensus.cc:2368] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:05.320566 30585 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:05.320700 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:05.331419 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 64008.9us
I20260709 13:11:05.331683 30530 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:05.331353 30585 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:05.332242 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.332892 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.333626 29009 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.307191 30549 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:11:05.338765 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.340307 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.340934 29151 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:05.341125 29083 raft_consensus.cc:2368] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:05.317497 30536 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:11:05.342010 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.343083 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.302769 30577 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.327548 29712 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:05.329300 30567 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.347937 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:05.348685 30397 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.349393 30397 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:05.350785 30397 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:05.351156 30397 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:05.352386 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.656s (SENT)
I20260709 13:11:05.352803 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.351761 29085 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:05.355098 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.359769 29085 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:05.366737 29048 leader_election.cc:336] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.664s (SENT)
I20260709 13:11:05.367125 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.368628 29005 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:05.371536 29078 raft_consensus.cc:2410] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:11:05.371660 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.372565 30604 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:05.357285 29077 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:05.374562 29009 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:05.365746 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:05.386421 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:05.378098 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.375543 29079 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:05.385788 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.388517 29085 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.395756 29001 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:05.413136 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.386080 30567 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.414347 29079 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:05.415581 29002 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:05.418509 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.419389 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.419974 29001 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
W20260709 13:11:05.419646 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 52914.6us
I20260709 13:11:05.375419 30479 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:11:05.428170 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:05.430987 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:05.439952 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.440457 29717 raft_consensus.cc:2393] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:05.440667 29002 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.400554 30313 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.434719 29085 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
W20260709 13:11:05.436986 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.439277 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.443207 29081 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:05.452093 29001 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:05.455094 29009 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.456311 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:05.457361 29009 raft_consensus.cc:2410] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:05.458254 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.459091 29009 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.459566 30479 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:11:05.459749 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.462970 30510 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:05.463367 30510 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:05.464545 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.464967 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.465778 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:05.466347 29048 leader_election.cc:336] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.728s (SENT)
I20260709 13:11:05.414212 29007 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:05.459749 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.469837 29009 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.470255 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.471122 29085 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.424690 30577 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.478797 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.480304 29009 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:05.401021 30397 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:05.484754 30397 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.492544 28975 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:05.493968 28975 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:05.468928 29007 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:05.510236 29079 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:05.503189 28975 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:05.513994 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 76014.5us
W20260709 13:11:05.515244 29048 leader_election.cc:336] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.738s (SENT)
I20260709 13:11:05.516176 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.517720 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:05.522914 29085 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:05.514273 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.531045 29085 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:05.513648 30605 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:05.533243 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:05.533886 29079 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:05.468446 30313 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.503798 30391 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:05.538852 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.535857 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:05.512230 30451 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:11:05.537708 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.512470 30594 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:05.543354 29002 raft_consensus.cc:2368] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4.
W20260709 13:11:05.545970 29048 leader_election.cc:336] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.770s (SENT)
I20260709 13:11:05.546505 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.530827 30577 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:05.551050 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 112580us
I20260709 13:11:05.557709 29156 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:05.561386 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.563709 29009 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.503624 30510 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.550554 30391 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:05.546274 30605 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:05.576200 29079 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:05.577734 29007 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.581833 29007 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.583376 30391 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.584466 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.585067 29001 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
W20260709 13:11:05.580991 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.591s (SENT) [suppressed 43 similar messages]
I20260709 13:11:05.581422 30605 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.585175 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.564033 30567 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:05.551607 30594 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:05.590977 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.588622 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:05.596652 29001 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.588773 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.596688 29079 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
W20260709 13:11:05.604028 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.617373 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.619746 29009 raft_consensus.cc:2368] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:05.620570 29079 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:05.588402 30510 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.621965 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.622613 29002 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 1.
I20260709 13:11:05.590569 29155 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:05.628633 30621 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:05.629022 30626 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 13:11:05.630044 30622 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:11:05.586156 29683 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:05.634086 30594 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:05.638552 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.615275 30577 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:05.646711 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.648793 29009 raft_consensus.cc:2368] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:05.652707 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:05.653398 29710 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:05.654964 30577 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:05.657008 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:05.657245 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.657860 29083 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:05.659659 29079 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:11:05.639051 29048 leader_election.cc:336] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.719s (SENT)
I20260709 13:11:05.661502 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.671133 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:05.672106 29150 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.673717 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.677404 29001 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:05.685617 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:05.676249 30626 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:05.688366 30626 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:05.688560 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.689391 29002 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.689715 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.689888 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:05.691501 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:05.688746 30626 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.691864 29081 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:05.691810 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:05.692453 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 118672us
W20260709 13:11:05.697324 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58318.2us
I20260709 13:11:05.698364 29081 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:11:05.698547 29048 leader_election.cc:336] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.878s (SENT)
I20260709 13:11:05.698936 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.686178 29083 raft_consensus.cc:2376] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3. Re-sending same reply.
W20260709 13:11:05.703833 29048 leader_election.cc:336] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.891s (SENT)
I20260709 13:11:05.699632 30567 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.704982 30594 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:05.705199 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.705724 30536 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:11:05.706259 30567 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:05.706583 29048 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.749s (SENT)
I20260709 13:11:05.707026 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.708210 30567 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.708117 30509 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:05.709847 30506 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.710319 30506 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:05.711838 30577 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:05.712601 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:05.676872 30451 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:05.715022 30451 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.716903 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:05.708258 29048 leader_election.cc:336] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.944s (SENT)
I20260709 13:11:05.717690 29084 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:05.718393 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:05.722185 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.722635 30506 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:05.723127 29004 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.692544 30494 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.687192 30396 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:05.685782 30383 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.729094 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:05.730762 30313 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:05.731173 30550 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:05.731509 30594 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.747586 30565 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.733445 29007 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:11:05.737833 29159 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
W20260709 13:11:05.743366 29048 leader_election.cc:336] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.024s (SENT)
I20260709 13:11:05.726243 30598 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.753245 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.740517 30549 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.763861 30625 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:05.765798 29048 leader_election.cc:336] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.042s (SENT)
I20260709 13:11:05.766028 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:05.766245 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:05.759708 30626 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:05.761476 30549 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.769207 30335 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:05.769731 30335 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:05.770476 30626 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:05.770646 30335 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:05.771042 30549 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:05.771952 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.772698 30335 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.755679 29007 pending_rounds.cc:85] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 1
I20260709 13:11:05.775758 29154 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:05.776078 29007 pending_rounds.cc:107] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:11:05.777158 29007 raft_consensus.cc:2889] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:05.778000 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:05.778908 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:05.768546 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.784404 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:05.785336 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:05.785575 29001 raft_consensus.cc:2376] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3. Re-sending same reply.
I20260709 13:11:05.785933 29083 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:05.782330 30625 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.768298 30577 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.789299 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.789996 29083 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:05.790652 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.791240 29083 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:05.782182 30494 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:05.792873 29048 leader_election.cc:336] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.917s (SENT)
W20260709 13:11:05.793860 29048 leader_election.cc:336] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.064s (SENT)
I20260709 13:11:05.794024 29714 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:05.794241 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.795466 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.797348 29083 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:05.800017 30510 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:05.809525 29125 leader_election.cc:400] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:05.810053 29125 leader_election.cc:403] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:05.814790 30396 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.815133 28976 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:05.815384 30396 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.815948 30605 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.818701 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.808389 30494 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.820839 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.821096 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.821527 29001 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:05.795265 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.817035 29004 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.825953 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:05.823992 30625 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:05.827453 29711 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:05.827677 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.828338 30594 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.828451 29004 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.826598 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:05.829246 29076 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.829736 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.829993 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:05.830089 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.830700 30596 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:05.830911 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.831482 29010 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:11:05.832188 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.828445 30396 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:05.832410 29004 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:05.832957 30596 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.834230 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.835557 30584 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:05.841270 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45109.9us
I20260709 13:11:05.845448 29048 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.844686 29083 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:05.848726 29125 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:05.858547 30596 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:05.859227 30596 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.847100 29076 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:05.859705 30596 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.858539 30510 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:05.861585 30510 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:05.860893 29076 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:05.864259 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.864368 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:05.846571 30584 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.869051 30584 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:05.869397 30510 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.871273 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:05.873160 29010 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:05.875372 29080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 140) took 1346 ms (client timeout 1751 ms). Trace:
W20260709 13:11:05.875775 29080 rpcz_store.cc:269] 0709 13:11:04.529419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:04.546504 (+ 17085us) service_pool.cc:224] Handling call
0709 13:11:05.875161 (+1328657us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b5400148a54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:05.875293 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733676800}
I20260709 13:11:05.879328 28907 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51350 (request call id 146) took 6828 ms. Trace:
I20260709 13:11:05.884938 29048 leader_election.cc:400] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
W20260709 13:11:05.912470 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55493.6us
I20260709 13:11:05.914242 29048 leader_election.cc:403] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:05.909241 29083 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:05.916635 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:05.926165 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.846822 29007 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
W20260709 13:11:05.931344 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 78354.7us
I20260709 13:11:05.932523 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.928165 30549 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:05.936525 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 89534.6us
I20260709 13:11:05.937572 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:05.940292 30549 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.941171 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:05.941987 29711 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:05.942160 30598 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.941987 29125 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:05.943723 30510 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:05.943894 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:05.944025 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:05.957856 30395 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.958956 29082 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:05.959215 30637 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:05.936193 30396 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:05.959970 30396 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:05.960402 30396 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.926852 30577 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:05.912451 30383 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:11:05.956012 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:05.940378 30328 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.965515 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:05.944265 30335 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:05.925822 30626 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:11:05.973847 30565 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:05.980818 30565 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:05.960876 30479 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:05.981920 30335 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:05.882017 28907 rpcz_store.cc:276] 0709 13:10:59.050902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.051016 (+   114us) service_pool.cc:224] Handling call
0709 13:11:05.879228 (+6828212us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:04.818270 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:04.896992 (+ 78722us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:04.927678 (+ 30686us) write_op.cc:435] Acquired schema lock
0709 13:11:04.927703 (+    25us) tablet.cc:662] Decoding operations
0709 13:11:05.052285 (+124582us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:05.071573 (+ 19288us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:05.073745 (+  2172us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:11:05.148561 (+ 74816us) tablet.cc:701] Row locks acquired
0709 13:11:05.148610 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:11:05.148788 (+   178us) write_op.cc:270] Start()
0709 13:11:05.148938 (+   150us) write_op.cc:276] Timestamp: P: 1783602665148727 usec, L: 0
0709 13:11:05.148976 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:11:05.185488 (+ 36512us) log.cc:844] Serialized 2050 byte log entry
0709 13:11:05.584314 (+398826us) op_driver.cc:464] REPLICATION: finished
0709 13:11:05.680660 (+ 96346us) write_op.cc:301] APPLY: starting
0709 13:11:05.680786 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:05.681011 (+   225us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:05.681230 (+   219us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:05.682524 (+  1294us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:05.682973 (+   449us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:11:05.683065 (+    92us) write_op.cc:312] APPLY: finished
0709 13:11:05.683531 (+   466us) log.cc:844] Serialized 47 byte log entry
0709 13:11:05.683765 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:05.683908 (+   143us) write_op.cc:454] Released schema lock
0709 13:11:05.683977 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:05.684503 (+   526us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:05.684548 (+    45us) write_op.cc:454] Released schema lock
0709 13:11:05.773648 (+ 89100us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b54000f7dd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583e32a 00007f41157c3f14 00007f41157c3ec3 00007f41157c3e38 00007f41157c3df3 00007f41157c30dc 00007f41157ece16 00007f41157d4cbb 00007f41157bf39d 00007f41157c2d75
Metrics: {"mutex_wait_us":286910,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":96013,"apply.run_cpu_time_us":3791,"apply.run_wall_time_us":3799,"num_ops":5,"prepare.queue_time_us":474112,"prepare.run_cpu_time_us":38679,"prepare.run_wall_time_us":547505,"raft.queue_time_us":125940,"raft.run_cpu_time_us":68744,"raft.run_wall_time_us":191059,"replication_time_us":435194,"spinlock_wait_cycles":3143040,"thread_start_us":225134,"threads_started":4,"wal-append.queue_time_us":202539}]]}
I20260709 13:11:05.995813 30335 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:05.964779 30596 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:05.998979 30596 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:05.952234 30573 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:11:06.001467 28908 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52676 (request call id 241) took 8469 ms. Trace:
I20260709 13:11:05.977473 30510 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:06.001852 28908 rpcz_store.cc:276] 0709 13:10:57.532247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.532425 (+   178us) service_pool.cc:224] Handling call
0709 13:11:06.001274 (+8468849us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":5}
I20260709 13:11:06.002408 30510 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.002815 30510 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:06.007993 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:06.009414 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:05.988925 30598 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:05.948719 29083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "243d4ae4a4ae48b2979e32874472d256"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:06.011188 29083 raft_consensus.cc:606] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Received request to transfer leadership to TS 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.011544 29083 raft_consensus.cc:614] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:06.012060 29714 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:11:06.012203 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.027980 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.029312 28975 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:06.027908 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:06.040215 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 77882.7us
I20260709 13:11:06.026443 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:06.047153 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:06.025709 30637 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:06.048713 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 83293.8us
I20260709 13:11:06.052429 30648 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:06.040021 30395 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:06.033274 30605 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.055235 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.056228 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.057818 30577 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:06.064085 30577 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:06.065101 30577 raft_consensus.cc:3055] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:06.065569 30577 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:06.065201 30584 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.077320 29158 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:06.040391 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:06.081046 29048 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.083592 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:06.084170 29158 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.085517 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:06.055189 29711 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.086139 29083 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:06.057914 30573 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:06.094621 29714 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:06.102031 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:06.068599 30634 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:06.104190 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:06.084077 30451 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:06.116914 29080 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:06.086218 30584 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:06.118324 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:06.118954 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:06.120440 29084 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.085963 30650 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:06.069730 30637 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.125099 29158 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:06.086848 30565 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
W20260709 13:11:06.120450 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 91202us
W20260709 13:11:06.131616 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 78234.4us
I20260709 13:11:06.133508 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:06.066803 30577 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.118333 30573 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.137326 29716 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:06.141175 29716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 270) took 1003 ms. Trace:
I20260709 13:11:06.142628 29716 rpcz_store.cc:276] 0709 13:11:05.137686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:05.137946 (+   260us) service_pool.cc:224] Handling call
0709 13:11:06.091597 (+953651us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:06.141119 (+ 49522us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22057856}
I20260709 13:11:06.132853 30397 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:11:06.142182 30573 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.159114 30665 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.146571 30395 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:06.204775 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:06.206748 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1263) took 1255 ms. Trace:
I20260709 13:11:06.207144 29082 rpcz_store.cc:276] 0709 13:11:04.951786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:04.951896 (+   110us) service_pool.cc:224] Handling call
0709 13:11:06.206555 (+1254659us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54001accd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:06.206694 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2109124992}
W20260709 13:11:06.209144 30654 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:11:06.218034 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 131229us
I20260709 13:11:06.218946 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.201274 30550 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.220422 30335 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.220795 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.220928 30335 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.143576 30660 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:11:06.222010 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.222841 30669 raft_consensus.cc:2764] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:11:06.223407 30660 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:06.223502 30669 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:06.223639 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.223961 30669 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:06.224418 30669 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:06.224781 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:06.221410 30653 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:11:06.215235 30451 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:06.230656 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.223891 30660 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:06.244287 30649 log.cc:927] Time spent T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:11:06.258396 30637 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:06.258915 30671 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:06.254377 30668 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:06.247222 30335 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:06.260322 30395 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.227052 30550 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.264578 30659 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:06.264642 30549 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:06.266868 30648 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:06.275342 30671 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:06.281704 29084 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:06.270766 30660 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.287458 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.289639 29158 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:06.287420 30659 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:06.263381 30451 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.296705 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:06.304731 29002 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:06.300068 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:06.300150 29048 leader_election.cc:336] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.169s (SENT)
I20260709 13:11:06.301149 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:06.306394 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.306672 29001 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:06.302938 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:06.310832 29048 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.239s (SENT)
I20260709 13:11:06.311450 29152 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:06.310222 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:06.297688 30668 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:06.311964 29152 raft_consensus.cc:606] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:06.314154 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 186722us
I20260709 13:11:06.296012 30671 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.317281 29085 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:06.317960 29006 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:06.318372 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:06.318949 29082 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:06.302150 30659 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.315197 29003 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:06.327508 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:06.332387 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:06.312453 30397 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:06.332474 29009 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:06.314729 30604 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.343307 29001 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:06.329413 30335 raft_consensus.cc:2764] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:11:06.353610 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:06.375000 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 75023.3us
I20260709 13:11:06.336238 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:06.339071 30396 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.344900 30573 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:06.331691 30479 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:11:06.353279 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:06.389729 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:06.374364 29008 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:06.397702 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:06.378026 30675 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:06.403033 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:06.413779 30550 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:06.378054 30604 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.415162 30510 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:06.419580 30657 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:11:06.431095 29711 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:06.420491 30396 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:06.424712 29159 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:06.425079 29153 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:06.433732 30669 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.425390 29152 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:06.422048 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:06.436580 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.437479 30396 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:06.438195 30396 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:06.438613 30396 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.418996 30605 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:06.430145 28906 catalog_manager.cc:5697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 2 to 3, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.441717 30671 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.443244 29010 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:06.444023 30671 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:06.443819 30659 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.445415 30669 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:06.445513 30604 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:06.452692 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63060.7us
I20260709 13:11:06.454180 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:06.446979 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 109727us
I20260709 13:11:06.440219 30598 raft_consensus.cc:2764] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:11:06.464813 28906 catalog_manager.cc:5697] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.479928 30397 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.480528 30397 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.482522 30397 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:06.483518 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:06.426376 30510 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:06.485333 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:06.484218 29151 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:06.488692 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:06.486552 28906 catalog_manager.cc:5697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 2, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:06.484519 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:06.448271 30659 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:06.492143 28906 catalog_manager.cc:5697] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 2, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 2 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.494405 30672 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.494935 30672 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.493944 28906 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 2 to 3, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.501361 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:06.503803 30683 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:06.501785 28906 catalog_manager.cc:5697] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 2 to 3, term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to <none>, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:06.504293 30683 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.505585 30683 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:06.505901 28906 catalog_manager.cc:5697] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.507094 30669 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:06.507519 28906 catalog_manager.cc:5697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 2 to 4, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 4 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.485896 30267 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:06.508734 30267 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:06.509106 30267 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.509217 28906 catalog_manager.cc:5697] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 2, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 2 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:06.491729 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 55978.1us
I20260709 13:11:06.514005 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:06.474145 30479 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:06.514575 29079 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:06.490162 30550 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:11:06.517441 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:06.517985 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:06.518692 30548 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:06.519059 30548 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.519389 30548 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.491317 30655 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:06.520355 30655 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.520674 30655 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:06.523624 30510 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:06.526713 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.528364 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.529088 30659 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:06.529426 30659 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:06.529748 30659 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.533205 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:06.533836 29151 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
W20260709 13:11:06.533036 29078 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1267) took 1460 ms (client timeout 1803 ms). Trace:
W20260709 13:11:06.545792 29078 rpcz_store.cc:269] 0709 13:11:05.052613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:05.052818 (+   205us) service_pool.cc:224] Handling call
0709 13:11:06.512475 (+1459657us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b540034e754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:06.512659 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706857216}
W20260709 13:11:06.587131 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51577.6us
I20260709 13:11:06.588240 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:06.589491 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 54340.4us
I20260709 13:11:06.531112 30548 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.595162 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:06.596822 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:06.602697 30605 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:06.603211 30605 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:06.603303 30672 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:06.603633 30605 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.595798 30584 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:11:06.594007 30549 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:06.585218 30669 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:06.607530 30669 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:06.607919 30669 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:06.608203 30663 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:06.608599 30663 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.608899 30663 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:06.612294 30681 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:06.617606 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:06.618245 29158 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:06.620466 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:06.623123 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.619323 30494 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.627938 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:06.629096 29079 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:06.628104 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:06.631470 29078 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:06.631374 28976 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:06.639623 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:06.618835 30510 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:06.641374 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 140080us
I20260709 13:11:06.641482 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:06.629364 30565 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:06.625696 30549 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:06.646341 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:06.652691 29079 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:06.643846 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:06.645372 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.657418 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:06.661779 29122 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:06.647934 30391 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:06.629110 30671 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:11:06.632499 30681 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:11:06.643576 30451 raft_consensus.cc:2764] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:11:06.665503 30682 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:06.640820 30678 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:06.648234 30565 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.635514 30494 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.626425 30634 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:11:06.649186 30680 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:06.661084 30598 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:06.685777 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:06.674443 29077 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:06.674609 30313 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:06.680886 30682 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:06.694796 30395 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:06.691836 30565 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.690773 29154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "824fc688db264856b025a29c651eec78"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:06.724503 29154 raft_consensus.cc:606] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:06.692672 30391 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:06.728402 30391 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.715821 30678 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:06.728921 30391 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:06.737751 30396 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.739805 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:06.741684 29002 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:06.701592 30655 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.674499 30550 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:06.648604 30510 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.705060 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:06.695353 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 452) took 1751 ms. Trace:
I20260709 13:11:06.701916 30549 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.715416 30395 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:06.715613 30682 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:06.693248 30596 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.760857 28976 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.711258 30598 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.673405 30683 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.721710 30548 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:06.733665 30605 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.793532 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:06.785097 30634 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:06.785787 30395 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.786815 30669 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.790066 30335 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:06.752424 30313 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:06.783584 30596 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.772521 30396 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:06.796311 30391 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.800158 30672 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.745242 30659 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.784010 30598 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.799034 30494 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:06.802873 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.799536 30335 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:06.799095 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:06.836925 30548 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.837641 30548 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.844317 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:06.844967 29082 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:06.847296 30577 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.855809 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:06.857547 30648 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:06.857159 29153 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.858011 30648 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.858420 30648 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.862850 30548 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:06.804754 30395 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:06.893801 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:06.813519 30605 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:06.810340 30634 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:06.806476 30669 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:06.803519 30678 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:06.812636 30510 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:06.904268 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:06.917667 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 115882us
I20260709 13:11:06.934788 29008 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:06.937402 30596 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:06.802350 30625 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:11:06.885344 30681 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:06.944904 29009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
mode: GRACEFUL
new_leader_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:47940
I20260709 13:11:06.946643 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:06.812965 30585 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:11:06.952363 29009 raft_consensus.cc:606] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Received request to transfer leadership to TS ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:06.803473 30660 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.955488 29009 raft_consensus.cc:614] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:06.825989 30672 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:06.959395 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:06.964001 29080 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:06.764468 29708 rpcz_store.cc:276] 0709 13:11:04.944091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:04.948384 (+  4293us) service_pool.cc:224] Handling call
0709 13:11:05.712328 (+763944us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:05.712562 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:05.761171 (+ 48609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:05.761204 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:05.761274 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:05.761307 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:06.694971 (+933664us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b540048f154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:06.695068 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:06.695273 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124524672}
I20260709 13:11:06.896301 30267 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:06.977159 29158 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:06.979600 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:07.017839 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:07.022869 29159 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:06.994812 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:07.026423 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:06.960552 30681 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:07.030838 29158 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:06.968271 30669 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:07.045580 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:07.018538 30548 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:07.057376 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:07.073232 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 191533us
I20260709 13:11:07.035298 30680 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:06.973765 30313 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.035869 30634 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.143168 30396 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:11:07.183487 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:07.020506 30663 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.019822 30678 consensus_queue.cc:1059] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.145s
I20260709 13:11:07.142948 30328 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.028312 29005 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:07.197577 29083 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:06.975620 30335 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.200912 29080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "ca45420e033847489ad989c1c6eed3ef"
 from {username='slave'} at 127.0.0.1:42002
W20260709 13:11:07.179999 30696 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:11:07.176010 29008 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:07.184203 30550 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:07.200171 30651 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:07.205667 30577 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:07.219960 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 472818us
I20260709 13:11:07.385910 29051 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:07.226774 29006 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:07.248864 29080 raft_consensus.cc:606] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Received request to transfer leadership to TS ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:07.389531 29080 raft_consensus.cc:614] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:11:07.300230 29048 leader_election.cc:336] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.591s (SENT)
W20260709 13:11:07.354741 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:07.250662 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:07.378505 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:07.397553 29150 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
W20260709 13:11:07.297561 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:07.372877 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 313020us
W20260709 13:11:07.394824 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 172130us
W20260709 13:11:07.408696 30692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.003s	sys 0.000s
I20260709 13:11:07.410772 29157 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:11:07.344538 30565 consensus_meta.cc:287] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.004s
W20260709 13:11:07.369426 30695 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:11:07.213238 30267 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:07.218968 30659 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:07.380461 30703 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:11:07.423888 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:07.211009 30680 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:07.420811 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:07.341702 30660 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:07.375341 30655 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:07.262074 30669 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:07.468978 30705 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:11:07.215512 30681 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.370465 30675 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:11:07.483227 29001 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.022s	sys 0.071s
I20260709 13:11:07.372845 30672 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:07.396065 30596 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:07.420856 30692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.003s	sys 0.000s
W20260709 13:11:07.496425 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 196232us
I20260709 13:11:07.210843 30682 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.492363 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:11:07.501902 29001 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.023s	sys 0.071s
I20260709 13:11:07.238961 30328 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.464104 30530 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:07.413054 30584 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.022s	sys 0.047s
I20260709 13:11:07.526481 30675 raft_consensus.cc:697] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:07.531735 30328 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:07.532327 30675 consensus_queue.cc:237] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:11:07.534369 30598 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.640s	user 0.005s	sys 0.003s
I20260709 13:11:07.534767 30598 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.380785 30704 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.146s
I20260709 13:11:07.537412 30682 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:07.464918 30565 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.538180 29156 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:07.539731 30565 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:07.367251 30651 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.343380 30648 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.396739 30577 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.469555 30700 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:07.469820 30664 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:07.206315 30548 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.539347 30669 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.517444 29158 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:07.540635 29079 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:07.375047 30550 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.567539 30577 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.540462 30663 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:07.565006 30395 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.629s	user 0.007s	sys 0.003s
I20260709 13:11:07.546005 30509 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.547251 30648 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:07.554636 30651 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.557523 30536 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:07.520852 30672 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.602818 30672 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.605456 30672 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:07.540799 30584 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.023s	sys 0.047s
I20260709 13:11:07.468132 30391 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.497270 30680 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:07.571167 30664 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.608551 30664 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.610674 30548 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.540947 30530 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:07.577162 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 69412.1us
I20260709 13:11:07.588842 30267 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.592389 30509 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.613978 30267 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.520309 30596 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.615621 30509 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.616219 30267 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:07.564694 30700 raft_consensus.cc:697] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:07.613322 30536 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.612928 30530 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.617347 30700 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.623288 30672 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.623826 30335 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:07.625957 30672 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.629498 30536 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.632113 30710 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.632642 30710 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.633436 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:07.634058 29708 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:07.635682 29125 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:07.624794 30596 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.623517 30664 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:07.639541 30585 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:07.643646 30451 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.007s	sys 0.010s
I20260709 13:11:07.639813 30672 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.646749 30598 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:07.634756 30710 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.660763 30267 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:07.661836 30683 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:07.662245 30267 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:07.662703 30267 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:07.664276 29153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.081s	sys 0.224s
W20260709 13:11:07.672873 29153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.082s	sys 0.224s
I20260709 13:11:07.654693 30585 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:07.657044 30451 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.007s	sys 0.010s
W20260709 13:11:07.685663 30711 log.cc:927] Time spent T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.187s	user 0.003s	sys 0.000s
I20260709 13:11:07.663585 30267 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.656351 30664 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.677137 30585 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:07.695289 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 486543us
I20260709 13:11:07.761024 30716 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.761719 30716 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.762864 29708 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:07.764304 30267 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:07.764575 30716 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:07.762372 30680 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:07.798184 29083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1296) took 1697 ms (client timeout 2069 ms). Trace:
W20260709 13:11:07.799341 29083 rpcz_store.cc:269] 0709 13:11:06.101161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.101715 (+   554us) service_pool.cc:224] Handling call
0709 13:11:07.797960 (+1696245us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400336b54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:07.798113 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2483611008}
W20260709 13:11:07.796640 30634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.025s	sys 0.041s
W20260709 13:11:07.827814 30634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.025s	sys 0.041s
I20260709 13:11:07.830654 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:07.831532 29717 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:07.830683 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:07.826354 30672 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:07.834100 30672 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.835709 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:07.836779 29710 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:07.837189 30585 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:07.841575 30680 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.839501 30672 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:07.826522 30396 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.012s	sys 0.027s
W20260709 13:11:07.849602 29152 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.022s	sys 0.022s
W20260709 13:11:07.851066 29152 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.022s	sys 0.022s
W20260709 13:11:07.806586 30668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.014s	sys 0.025s
W20260709 13:11:07.842511 30396 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.012s	sys 0.027s
I20260709 13:11:07.853895 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:07.857884 29682 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:07.868769 29081 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:07.869674 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:07.870003 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:07.870275 29082 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:07.870795 29081 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:07.872390 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:07.873304 29081 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:07.875588 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:07.876211 29085 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:07.854877 30668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.015s	sys 0.026s
I20260709 13:11:07.890888 29008 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:07.873903 30596 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:07.892439 30596 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.892253 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:07.854334 30585 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:07.892853 30596 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:07.893157 29708 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:07.894392 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:07.895148 29006 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:07.872452 30710 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:07.896363 30565 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:07.897239 30717 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:11:07.897012 30565 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:07.896267 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:07.899652 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:07.901430 29083 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:07.906055 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:07.906827 29078 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:07.908263 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:07.909070 29711 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
W20260709 13:11:07.915586 29151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.009s	sys 0.034s
W20260709 13:11:07.916724 29151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.010s	sys 0.034s
I20260709 13:11:07.919314 29085 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:07.877378 30396 consensus_queue.cc:1243] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:11:07.926302 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:07.927129 29002 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:07.888686 30663 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.929768 29051 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:07.930420 29081 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:07.929844 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:07.891790 30678 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.002s	sys 0.039s
I20260709 13:11:07.896425 30710 raft_consensus.cc:697] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:07.933101 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:07.889196 30651 consensus_queue.cc:1059] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:07.929354 30680 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:07.925751 30383 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:07.933620 30397 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:07.945160 29078 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:07.947126 29082 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
W20260709 13:11:07.920603 29077 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.020s	sys 0.045s
I20260709 13:11:07.953663 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:07.934019 30663 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:07.965133 29077 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.020s	sys 0.046s
W20260709 13:11:07.952530 30678 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.002s	sys 0.040s
W20260709 13:11:07.977380 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 465330us
I20260709 13:11:07.961587 30397 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:08.002022 29048 leader_election.cc:336] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.860s (SENT)
I20260709 13:11:08.024607 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:08.026904 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1377) took 1238 ms. Trace:
I20260709 13:11:08.027410 29001 rpcz_store.cc:276] 0709 13:11:06.739145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.764543 (+ 25398us) service_pool.cc:224] Handling call
0709 13:11:06.764844 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:06.765041 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:07.520345 (+755304us) log.cc:844] Serialized 14 byte log entry
0709 13:11:07.976319 (+455974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:07.976399 (+    80us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:07.976463 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:07.976565 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:07.977975 (+  1410us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":629731,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3312,"raft.queue_time_us":21951,"thread_start_us":1183340,"threads_started":2}
I20260709 13:11:07.964406 30672 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:11:07.977078 29076 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.037s	sys 0.040s
I20260709 13:11:07.967010 30383 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:08.056501 29076 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.038s	sys 0.040s
I20260709 13:11:08.052621 30672 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:08.059070 30383 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:07.955403 30506 consensus_queue.cc:1059] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:11:08.058960 30672 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.060447 30506 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:08.060626 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:08.048599 28976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.008s	sys 0.066s
I20260709 13:11:08.069209 29008 raft_consensus.cc:2410] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:11:08.069811 28976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.009s	sys 0.067s
W20260709 13:11:08.070557 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 1.01312e+06us
I20260709 13:11:08.073882 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:07.962409 30680 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.026103 30397 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:08.068776 30506 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:08.068799 30724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.004s	sys 0.031s
I20260709 13:11:08.112797 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:08.115130 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:08.110035 30672 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:08.156795 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:08.113633 30724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.005s	sys 0.031s
I20260709 13:11:08.116536 30681 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:08.122915 30678 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:08.158923 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:08.151543 30396 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:08.168913 29082 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:08.171613 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 216742us
W20260709 13:11:08.172704 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.174146 29082 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:08.176302 29078 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:08.178141 30719 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:11:08.178709 29082 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:08.172883 30577 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:08.182997 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.184468 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:08.185124 29711 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:08.188519 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:08.189978 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:08.190717 29710 raft_consensus.cc:2368] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3.
I20260709 13:11:08.191228 30680 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:08.188532 30727 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:11:08.194132 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:08.114925 30710 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.199406 30663 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.200575 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.201181 30663 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:08.201522 29710 raft_consensus.cc:2410] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:08.202023 30663 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.202488 30663 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.161620 30678 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:08.204881 30678 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:08.157088 30605 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:11:08.179646 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1017) took 1203 ms. Trace:
I20260709 13:11:08.190745 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1774) took 1134 ms. Trace:
I20260709 13:11:08.208860 29002 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:08.165305 30681 raft_consensus.cc:697] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:08.206611 29152 rpcz_store.cc:276] 0709 13:11:06.975971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.976364 (+   393us) service_pool.cc:224] Handling call
0709 13:11:07.020366 (+ 44002us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:07.022628 (+  2262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:07.022660 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:07.030443 (+  7783us) log.cc:844] Serialized 26 byte log entry
0709 13:11:07.851508 (+821065us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:07.851567 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:07.851663 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:08.179357 (+327694us) raft_consensus.cc:1685] finished
0709 13:11:08.179386 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:08.179562 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1907,"raft.queue_time_us":2189,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":13294,"thread_start_us":688195,"threads_started":1,"wal-append.queue_time_us":1024156}
I20260709 13:11:08.208664 29151 rpcz_store.cc:276] 0709 13:11:07.056730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:07.056846 (+   116us) service_pool.cc:224] Handling call
0709 13:11:07.057192 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:07.166066 (+108874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:07.166114 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:07.180856 (+ 14742us) log.cc:844] Serialized 26 byte log entry
0709 13:11:07.918017 (+737161us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:07.918073 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:07.918164 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:08.190564 (+272400us) raft_consensus.cc:1685] finished
0709 13:11:08.190583 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:08.190695 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83069,"raft.queue_time_us":85583,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":11219,"thread_start_us":693553,"threads_started":1,"wal-append.queue_time_us":912054}
I20260709 13:11:08.252180 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.261010 29078 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:08.170193 30506 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.283535 29078 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:08.284562 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.277797 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 135) took 1535 ms. Trace:
I20260709 13:11:08.286159 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 160) took 1443 ms. Trace:
W20260709 13:11:08.290567 30709 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.789s	user 0.005s	sys 0.003s
I20260709 13:11:08.292472 29077 rpcz_store.cc:276] 0709 13:11:06.842506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.843331 (+   825us) service_pool.cc:224] Handling call
0709 13:11:06.843735 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:06.940234 (+ 96499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:06.942458 (+  2224us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:07.018393 (+ 75935us) log.cc:844] Serialized 26 byte log entry
0709 13:11:07.406608 (+388215us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:08.062050 (+655442us) log.cc:844] Serialized 14 byte log entry
0709 13:11:08.285822 (+223772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:08.285903 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:08.285947 (+    44us) raft_consensus.cc:1685] finished
0709 13:11:08.285963 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:08.286102 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49979,"prepare.queue_time_us":707402,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":231736,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":227,"thread_start_us":982911,"threads_started":3,"wal-append.queue_time_us":535424}
I20260709 13:11:08.307384 30748 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:08.309258 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.312105 30681 consensus_queue.cc:237] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.312773 30671 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:08.287191 30663 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.315024 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.327831 29081 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:08.316866 29711 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:08.287357 29005 rpcz_store.cc:276] 0709 13:11:06.742023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.766700 (+ 24677us) service_pool.cc:224] Handling call
0709 13:11:06.771427 (+  4727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:07.389337 (+617910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:08.277545 (+888208us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400026054. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:08.277733 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482619776}
I20260709 13:11:08.331140 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.317636 30397 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.333925 29078 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:08.336519 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.326705 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1311) took 1697 ms. Trace:
I20260709 13:11:08.314873 30574 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:08.331305 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1015) took 1806 ms. Trace:
I20260709 13:11:08.337124 29081 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
W20260709 13:11:08.340718 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 144986us
I20260709 13:11:08.344095 29159 rpcz_store.cc:276] 0709 13:11:06.524636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.524895 (+   259us) service_pool.cc:224] Handling call
0709 13:11:06.525204 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:07.023405 (+498201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:08.331093 (+1307688us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b540013d154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:08.331252 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144012544}
I20260709 13:11:08.339942 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.338095 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.325901 30678 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.326597 30671 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:08.319164 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:08.348978 29080 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:08.351143 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.355177 28976 leader_election.cc:400] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3.
I20260709 13:11:08.352810 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.390012 28976 leader_election.cc:403] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:08.390484 29001 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
W20260709 13:11:08.391526 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 232681us
I20260709 13:11:08.326324 30748 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.347752 30391 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.425580 30391 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.425851 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:08.426661 29711 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:08.339553 30701 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:08.374662 29078 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:11:08.428081 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 73106.2us
I20260709 13:11:08.430581 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:08.410617 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:08.395287 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:08.394011 29048 leader_election.cc:336] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.110s (SENT)
I20260709 13:11:08.343931 30574 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:08.433319 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:08.424341 30397 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:08.434626 29048 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.963s (SENT)
I20260709 13:11:08.440497 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:08.400321 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.356391 30605 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:08.428977 30701 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:08.445375 28976 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:08.447063 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.432737 30396 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3.
I20260709 13:11:08.435073 30680 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:08.492369 30730 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:08.440253 30751 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:08.494532 30669 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:08.497080 29048 leader_election.cc:336] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.973s (SENT)
I20260709 13:11:08.497627 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:08.492149 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.492410 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.442679 29005 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:08.447991 30596 consensus_meta.cc:287] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.000s
I20260709 13:11:08.536686 29006 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:08.542409 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:08.546299 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:08.502071 30397 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:08.566183 29006 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:08.567134 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 113606us
W20260709 13:11:08.569428 29048 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 3.270s (SENT)
I20260709 13:11:08.569939 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:08.540555 30504 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:08.541653 30751 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:08.538036 30680 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:08.414568 30671 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.543361 30750 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:08.561827 30673 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:08.568935 30663 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.569187 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.583293 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 157) took 1906 ms. Trace:
I20260709 13:11:08.547072 30596 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.572062 30749 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:08.595489 30749 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.597641 30749 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.590148 30748 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.591084 29711 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:08.588032 29079 rpcz_store.cc:276] 0709 13:11:06.676931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.677088 (+   157us) service_pool.cc:224] Handling call
0709 13:11:08.583025 (+1905937us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b54000244d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:08.583214 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782493568}
I20260709 13:11:08.574821 30574 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.600339 29051 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:08.601155 30748 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:08.592672 30663 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.601624 30748 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:08.602497 30748 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.603163 30663 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.577237 30504 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.593369 30397 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:08.593650 30673 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.604632 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.605341 30504 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.606127 30673 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.607664 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.577991 30701 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.586402 30750 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.600401 30574 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:08.611177 30750 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.612704 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.594338 30680 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.588395 30396 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:11:08.447425 30605 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.621325 29006 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
W20260709 13:11:08.622257 29010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1385) took 1790 ms (client timeout 1666 ms). Trace:
W20260709 13:11:08.622938 29010 rpcz_store.cc:269] 0709 13:11:06.826864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.826997 (+   133us) service_pool.cc:224] Handling call
0709 13:11:08.616694 (+1789697us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54003ff554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:08.616810 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3955929856}
I20260709 13:11:08.623505 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.626075 29006 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:08.627552 30391 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:08.628432 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 59037.6us
I20260709 13:11:08.339236 29076 rpcz_store.cc:276] 0709 13:11:06.617503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.618669 (+  1166us) service_pool.cc:224] Handling call
0709 13:11:06.842942 (+224273us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ceee 0000000000375939 00007f411a7901e0 00007f411a8590c4 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:06.843133 (+   191us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:06.938107 (+ 94974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:06.941932 (+  3825us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:06.945171 (+  3239us) log.cc:844] Serialized 26 byte log entry
0709 13:11:07.408604 (+463433us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:08.057077 (+648473us) log.cc:844] Serialized 14 byte log entry
0709 13:11:08.274228 (+217151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:08.285557 (+ 11329us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:08.287334 (+  1777us) raft_consensus.cc:1685] finished
0709 13:11:08.304504 (+ 17170us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:08.315236 (+ 10732us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42705,"prepare.queue_time_us":748871,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":233246,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":15426,"spinlock_wait_cycles":17588224,"thread_start_us":1321577,"threads_started":3,"wal-append.queue_time_us":715523}
I20260709 13:11:08.630714 30605 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.631433 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.637357 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:08.638396 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:08.638779 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.639045 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:08.639747 30672 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:08.641175 30335 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:08.641709 30335 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.642053 30674 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:08.634069 30577 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:11:08.642624 30674 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.644392 30267 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:08.644730 29711 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:08.645117 30267 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.648253 30718 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.648404 30585 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.649112 30585 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:08.655392 30726 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.534s	user 0.001s	sys 0.000s
W20260709 13:11:08.658591 29007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1376) took 1919 ms (client timeout 1633 ms). Trace:
W20260709 13:11:08.659132 29007 rpcz_store.cc:269] 0709 13:11:06.738861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.762020 (+ 23159us) service_pool.cc:224] Handling call
0709 13:11:08.658291 (+1896271us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003f1bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:08.658496 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2802124544}
I20260709 13:11:08.660710 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.661516 29007 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:08.662647 30550 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.662765 30704 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:08.662704 30594 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.664963 30598 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.665446 30734 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:08.662626 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.668411 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.666710 30659 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:08.666846 30673 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:08.668625 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:08.665992 30734 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.674321 29710 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:08.663635 30704 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.675709 30734 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.676805 30704 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.664686 30594 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.679373 30594 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:08.637624 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.624115 30596 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.670027 29007 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:08.680323 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.680980 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.681247 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.681941 29007 raft_consensus.cc:2368] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4.
I20260709 13:11:08.682009 29010 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:08.670223 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.663586 30550 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.651744 30585 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:08.647742 30267 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:08.688683 30655 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.692206 30664 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.694893 30550 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.669183 30659 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:08.643821 30335 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:08.704698 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65705.4us
I20260709 13:11:08.705230 30651 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:08.705375 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:08.717290 30651 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:08.672567 30750 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:11:08.717836 30651 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.719048 29125 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:08.719231 30659 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.706014 30582 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:08.720386 30659 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:08.720263 30582 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.721796 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.722221 30582 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:08.722334 29008 raft_consensus.cc:2376] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5. Re-sending same reply.
I20260709 13:11:08.722852 30582 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:08.707235 30706 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.723560 30706 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.723937 30651 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.724406 30565 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.725634 30706 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.707683 30550 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:08.726292 30550 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.708143 30688 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:08.736331 30688 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.738922 30688 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.740053 30595 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:08.739970 30626 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:08.740547 30595 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:08.740929 30595 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:08.743024 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:08.743487 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.743921 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:08.744488 30716 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:08.744829 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:08.744969 30716 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.746143 29715 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:08.746330 30574 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:08.746940 30574 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:08.747121 30716 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.747783 30574 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.750164 30550 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.750725 30550 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:08.751173 30550 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:08.648804 30718 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.753258 29125 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:08.754298 30718 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.754822 30688 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:08.727267 30582 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:08.755795 30582 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.756453 30577 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:08.746183 30637 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:08.665467 30598 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:11:08.758805 30637 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.759200 30637 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:08.760468 30598 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:08.760732 30760 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:08.761281 30688 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:08.712334 30674 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.717409 30761 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:08.763053 30761 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:08.762503 30688 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.763870 30761 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.754467 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:08.765080 30761 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:08.765698 30674 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:08.766201 30674 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:08.767031 30674 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.713711 30536 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:08.768446 30536 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:08.772600 30706 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:08.773592 30550 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.776398 29076 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:08.777840 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.783463 29082 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:08.784447 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.785343 29082 raft_consensus.cc:2393] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:11:08.786046 29009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1387) took 1741 ms (client timeout 1766 ms). Trace:
I20260709 13:11:08.781257 29081 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:11:08.786568 29009 rpcz_store.cc:269] 0709 13:11:07.045119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:07.045331 (+   212us) service_pool.cc:224] Handling call
0709 13:11:08.785810 (+1740479us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400291b54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:08.785975 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2320400768}
I20260709 13:11:08.788233 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:08.790963 29009 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:08.792209 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.792541 29009 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:08.793768 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.794685 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.795300 29084 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:08.692090 30267 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:08.796991 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.814839 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:08.815660 29151 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:08.789242 30536 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.870105 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:08.878218 29152 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:08.790891 29710 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:08.889303 29008 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:08.793797 29005 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:08.908198 29007 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:08.920459 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.920753 29009 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:08.920394 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.922140 29083 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:08.925431 29009 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:08.871213 30267 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:08.936792 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 254404us
I20260709 13:11:08.937767 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:08.939311 29081 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:08.940433 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:08.922742 29077 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:08.946444 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:08.950942 29152 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:08.952282 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.953109 29083 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:08.947365 29153 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:08.948843 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:08.956893 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:08.957513 29156 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:08.934195 30396 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.944166 30680 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.965572 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:08.965891 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:08.966337 29152 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:08.934031 30605 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:08.940011 30763 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:08.967664 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:08.967753 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:08.960373 30675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.424s	user 0.001s	sys 0.033s
I20260709 13:11:08.970963 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:08.967149 30680 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:08.994796 30680 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:08.995151 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:08.996289 30675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.460s	user 0.001s	sys 0.033s
I20260709 13:11:08.996812 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.968161 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:08.997591 29714 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:08.997810 30763 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.000119 30605 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.002322 30605 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:09.004343 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.005285 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.005869 29083 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:08.969858 30396 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.006942 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:09.007467 29083 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:09.008153 30396 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:09.009383 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.009894 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.010490 29085 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:09.010653 30761 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:09.011234 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.011860 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.012621 29084 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:09.012849 30584 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:09.013411 29125 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.013345 30584 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.014453 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.005836 29077 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:09.015297 30584 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:09.015588 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:08.978053 29010 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:09.016829 30694 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:09.017544 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.018216 29080 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:09.016324 30718 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:09.018769 30718 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:09.019130 30718 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.020390 30479 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:09.020694 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.021000 30733 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.021569 30335 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:09.021912 30335 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:09.022265 30335 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:09.011163 30761 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.023638 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.024384 30761 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:09.026036 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:09.026578 30763 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:09.028128 30716 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:09.028641 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1021) took 2047 ms. Trace:
I20260709 13:11:09.028802 30716 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:09.029621 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.030649 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.031270 29007 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:08.990986 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.991775 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:08.976400 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:09.010126 29079 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:09.036211 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:09.020766 30479 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:09.028923 29151 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:08.977121 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.043300 29006 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:09.028680 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.000165 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.044325 29002 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:09.016845 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.044879 29010 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:08.990062 30267 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:08.990284 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.033881 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.034237 29080 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:09.013549 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.031414 30504 consensus_queue.cc:1059] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:09.015825 30584 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:09.041563 29709 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:09.041389 30595 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.042065 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.044135 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:09.045626 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.033429 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.046329 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.051296 29083 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:09.061990 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.030162 30716 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:09.224535 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.129441 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.136404 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:09.074988 30536 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:09.225823 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:09.183702 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.228830 29151 raft_consensus.cc:2410] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:09.144440 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.144555 30669 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 13:11:09.069286 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:09.127547 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60143.2us
I20260709 13:11:09.143136 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.227051 29002 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:09.230245 29006 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.067806 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.231017 29003 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
W20260709 13:11:09.224720 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 188568us
I20260709 13:11:09.234401 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:09.226697 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 182641us
I20260709 13:11:09.240815 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.241407 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.241506 29711 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:09.236163 30335 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.242062 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.242209 29157 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:09.244903 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:09.243507 30536 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:09.320359 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65303.1us
I20260709 13:11:09.335543 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.321733 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.248327 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:09.323987 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 77121.9us
I20260709 13:11:09.337466 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:09.357580 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:09.059753 30761 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:09.171358 30637 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.143653 30504 raft_consensus.cc:3055] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:11:09.278527 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.332473 29153 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:09.334517 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.048331 30479 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:09.340071 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:09.345498 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.394960 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.395845 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.393626 29156 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.346124 29006 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:09.347882 29003 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:11:09.397656 29003 pending_rounds.cc:85] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 1
I20260709 13:11:09.397902 29003 pending_rounds.cc:107] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:11:09.398119 29003 raft_consensus.cc:2889] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:09.066926 30584 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:09.235888 30605 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:09.359364 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:09.393399 30577 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:09.405929 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46630us
I20260709 13:11:09.406071 30577 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.406654 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:09.407480 30763 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:09.408141 30763 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:09.408195 30396 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:09.408694 29081 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:11:09.409850 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:09.411186 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.411437 30577 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:09.411911 30577 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:09.049453 29005 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:09.335494 29155 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:09.419415 30595 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:09.420583 30637 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:09.423635 30335 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:09.426244 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:09.427433 29156 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:09.427577 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:09.387913 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.338382 30536 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.370625 30675 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:09.368142 30730 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:09.487355 29125 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:09.398429 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.341099 30659 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:09.401923 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.335564 30585 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:11:09.388835 30504 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:09.029085 29154 rpcz_store.cc:276] 0709 13:11:06.980767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:07.016814 (+ 36047us) service_pool.cc:224] Handling call
0709 13:11:07.215084 (+198270us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ceee 0000000000375939 00007f411a7901e0 00007f411a8590c4 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:07.215268 (+   184us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:07.558307 (+343039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:07.558345 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:07.558811 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:11:08.033762 (+474951us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:08.067899 (+ 34137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:09.028329 (+960430us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54000956d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:09.028386 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:09.028438 (+    52us) raft_consensus.cc:1685] finished
0709 13:11:09.028449 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:09.028585 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28322,"raft.queue_time_us":30584,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6417,"spinlock_wait_cycles":911153152,"thread_start_us":416805,"threads_started":1,"wal-append.queue_time_us":607147,"wal-append.run_cpu_time_us":14098,"wal-append.run_wall_time_us":1140764}
I20260709 13:11:09.412132 30584 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:09.242144 30710 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:09.137933 30680 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.213s
I20260709 13:11:09.412245 30735 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:09.431576 30577 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.414254 30605 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:11:09.418207 30396 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:09.436189 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.391693 30761 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.428589 30391 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
W20260709 13:11:09.536224 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 302176us
I20260709 13:11:09.450731 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.539050 29156 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:09.539364 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 296157us
W20260709 13:11:09.337353 30688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.005s	sys 0.017s
I20260709 13:11:09.415984 29153 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:09.530279 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:09.531147 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 294866us
I20260709 13:11:09.523867 30697 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:09.524464 30710 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:09.546299 29151 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:09.547719 29153 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:09.542295 30688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.005s	sys 0.019s
I20260709 13:11:09.548323 30710 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:09.546432 30504 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.587040 30651 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:09.588959 30577 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:09.591291 30761 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:09.527712 30675 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
W20260709 13:11:09.598917 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:09.619117 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.553177 30335 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:09.624457 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.625728 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.552639 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 137) took 1670 ms. Trace:
I20260709 13:11:09.543532 30735 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:11:09.595366 30718 consensus_meta.cc:287] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.000s
I20260709 13:11:09.554013 30697 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:09.632619 30506 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.635676 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.636333 30735 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:09.638823 29078 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:09.536605 30396 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:09.648339 30585 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.649385 30585 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:09.649751 30585 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:09.650521 30585 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.652357 30585 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.652724 30574 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.653649 30697 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.639583 30718 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.638096 29154 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:09.658083 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 336261us
I20260709 13:11:09.642355 30504 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:09.620880 30651 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.673152 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.676654 29714 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:09.677812 29051 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:09.644930 30733 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:09.718840 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 303709us
I20260709 13:11:09.721364 29157 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:09.722295 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:09.723151 29003 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.630618 29004 rpcz_store.cc:276] 0709 13:11:07.879016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:07.882343 (+  3327us) service_pool.cc:224] Handling call
0709 13:11:07.928345 (+ 46002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:08.154494 (+226149us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:08.171749 (+ 17255us) log.cc:844] Serialized 14 byte log entry
0709 13:11:08.392791 (+221042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:08.405166 (+ 12375us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:08.408333 (+  3167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:09.538460 (+1130127us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54003e21d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:09.540327 (+  1867us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:09.549030 (+  8703us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8545,"prepare.queue_time_us":7203,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":1653,"raft.queue_time_us":44567,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5100,"spinlock_wait_cycles":427529600,"thread_start_us":135725,"threads_started":1,"wal-append.queue_time_us":239839,"wal-append.run_cpu_time_us":1261,"wal-append.run_wall_time_us":1009163}
I20260709 13:11:09.725296 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.725901 29006 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:09.658612 30396 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:09.735543 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.736387 29008 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:09.757017 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:09.757616 29002 raft_consensus.cc:2376] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3. Re-sending same reply.
I20260709 13:11:09.759831 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:09.762086 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 231751us
I20260709 13:11:09.778925 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:09.717012 30313 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:11:09.755410 29155 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:09.788561 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.789162 29006 raft_consensus.cc:2376] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5. Re-sending same reply.
I20260709 13:11:09.782403 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.728700 30267 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:11:09.787133 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.791790 29002 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:09.759514 30786 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:09.727743 30733 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:09.793334 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 115629us
I20260709 13:11:09.789186 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:09.792691 29008 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:11:09.804035 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.771718 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.758376 30688 consensus_queue.cc:1059] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:11:09.811820 30789 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.813234 30789 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.814538 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.816982 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.817835 29006 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:09.813997 30789 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:09.790747 30674 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:09.820514 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:09.821321 29008 raft_consensus.cc:2376] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5. Re-sending same reply.
I20260709 13:11:09.790143 30780 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:09.824146 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:09.729974 30735 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.806282 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:09.829824 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 157531us
I20260709 13:11:09.832651 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.833647 29008 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:09.826949 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.841962 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.847071 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:09.831534 29004 raft_consensus.cc:2393] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:09.824882 30335 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:09.850838 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.851610 29006 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:09.828173 30780 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:09.831717 30313 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:09.820001 30789 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:09.852067 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 47264.7us
I20260709 13:11:09.875926 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.841603 30688 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:09.873453 30335 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.874140 30780 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.882365 30678 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:09.897029 30678 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:09.897737 30584 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.898844 30651 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:09.899376 30785 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:11:09.899447 30335 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:09.850150 30735 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:09.900481 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 76446.4us
W20260709 13:11:09.901309 30787 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:11:09.898875 29158 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:09.807922 30786 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.894575 30795 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:09.907832 29003 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:09.845077 30536 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:09.909659 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 62662.6us
W20260709 13:11:09.911305 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 69418.7us
I20260709 13:11:09.915215 30786 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:09.915335 30536 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:09.915120 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:09.853722 30391 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:11:09.919147 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:09.919724 29078 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.922451 30710 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.924259 30801 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:09.924717 30801 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:09.924767 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.925158 29048 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:09.925676 29083 raft_consensus.cc:2410] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:09.879410 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.925195 30801 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:09.926908 29155 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:09.927505 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.927836 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.928988 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.930017 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:09.926590 30796 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:11:09.931311 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:09.931275 30780 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:09.934862 29078 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:09.937741 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:09.938309 29715 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:09.939556 30799 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:09.939971 30799 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:09.940322 30799 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:09.945130 29715 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:09.935425 30801 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:09.946394 29051 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:09.936106 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:09.937023 29048 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:09.937405 30780 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:09.949524 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:09.954524 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.953899 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:09.960172 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 46379.5us
I20260709 13:11:09.961689 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.962409 29714 raft_consensus.cc:2393] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:09.905416 30788 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:11:09.959532 29715 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 1 LEARNER]: Advancing to term 3
I20260709 13:11:09.961150 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.966173 30678 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:09.967012 30718 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:09.967453 30763 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:09.960088 29080 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:09.969617 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:09.970047 30791 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:11:09.973405 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:09.975744 29084 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:09.978432 29716 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:09.978981 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:09.979776 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:09.985339 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:09.989482 29078 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:09.990015 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:09.990363 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:09.959426 30780 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:09.995262 29716 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:09.968497 30651 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:09.968695 30688 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:10.000505 29717 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:09.968806 30267 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:09.976871 30391 consensus_queue.cc:1243] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:11:09.972990 30786 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:10.008289 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 71348.8us
W20260709 13:11:09.997874 30479 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.000s
I20260709 13:11:10.011082 29048 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:09.994451 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:09.980681 30313 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:09.981976 30678 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.003746 30718 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:10.030207 29048 leader_election.cc:400] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4.
I20260709 13:11:10.036361 29048 leader_election.cc:403] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:10.024554 30391 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:09.993891 30763 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.022058 30786 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.027393 30701 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:10.024235 30671 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:10.023710 30651 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:10.043675 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53744.5us
I20260709 13:11:09.972256 30789 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.068960 30789 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:10.069953 30577 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:10.048893 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:10.052371 30762 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:10.057469 30763 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:10.128185 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 167085us
W20260709 13:11:10.073953 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 43875.8us
I20260709 13:11:10.045256 30391 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.066743 30651 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.043710 30786 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:10.032975 30806 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:10.060487 30761 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:10.129213 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:10.037528 30718 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:10.130299 30762 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:10.073910 29154 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:10.074544 29007 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:10.129475 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:10.134243 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 138962us
I20260709 13:11:10.026820 30804 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:11:10.029237 30479 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.128458 30584 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:10.099962 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51176.3us
I20260709 13:11:10.099097 30801 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.133193 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:10.159700 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:10.163385 30763 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:10.163926 30763 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.164409 30763 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:10.166507 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:10.169203 29152 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:10.139510 30718 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:10.174439 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:10.151242 30762 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:10.157888 30813 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:10.140811 30811 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4.
I20260709 13:11:10.168890 30801 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:10.202529 30813 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:10.161017 30799 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.151526 30697 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:11:10.214711 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.216145 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.217350 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:10.217911 30733 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:10.220230 30396 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.221294 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 168) took 1214 ms. Trace:
I20260709 13:11:10.221776 29077 rpcz_store.cc:276] 0709 13:11:09.006430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.015796 (+  9366us) service_pool.cc:224] Handling call
0709 13:11:09.016091 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:10.132020 (+1115929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:10.133631 (+  1611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:10.136652 (+  3021us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:10.140019 (+  3367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:10.221030 (+ 81011us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400120554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:10.221083 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:10.221237 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2431812352}
I20260709 13:11:10.222710 30396 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:10.224637 30811 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:10.228762 30678 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.201136 30735 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:10.208702 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:10.228331 30762 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.205755 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.223706 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:10.266425 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 65076.8us
I20260709 13:11:10.268440 29084 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:10.267422 29159 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:10.218317 29708 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
W20260709 13:11:10.271437 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:10.271704 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 128569us
I20260709 13:11:10.273484 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:10.205046 30812 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 13:11:10.209614 30761 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:10.278376 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:10.211053 30813 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:10.265425 30804 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.283700 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:10.217942 29155 raft_consensus.cc:2376] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5. Re-sending same reply.
I20260709 13:11:10.267915 30735 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:10.273864 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:10.296175 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:10.296413 29003 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:10.296953 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:10.304175 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:10.274300 30783 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.281380 30823 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.281981 30710 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.296631 30810 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:10.281237 30678 consensus_queue.cc:1059] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
W20260709 13:11:10.424352 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 126668us
I20260709 13:11:10.281096 30396 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:11:10.314301 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:10.397810 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 93796.8us
I20260709 13:11:10.465029 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:10.324359 30783 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.475732 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:10.311899 30804 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.409750 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.376212 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:10.411803 30391 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.424981 30793 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.486531 29002 raft_consensus.cc:2410] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:10.411695 30810 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:10.471170 30805 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.478941 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:10.483052 30479 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:10.490090 30735 raft_consensus.cc:3055] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:10.490342 30810 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:10.488705 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.489943 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:10.511264 29716 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:10.494009 30391 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.493706 30830 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:10.507429 30798 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:10.505203 30808 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.508353 30735 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:10.527985 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:10.541354 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:10.482028 30761 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.545166 30801 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:10.509745 30827 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:10.526677 30831 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:11:10.508807 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 482) took 1482 ms. Trace:
I20260709 13:11:10.543560 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:10.557097 29005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1405) took 1583 ms (client timeout 1641 ms). Trace:
I20260709 13:11:10.557184 30396 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:10.557575 29005 rpcz_store.cc:269] 0709 13:11:08.973821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.014227 (+ 40406us) service_pool.cc:224] Handling call
0709 13:11:10.556920 (+1542693us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b54002053d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:10.557039 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42840832}
I20260709 13:11:10.557833 30396 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.517558 30799 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.556556 29711 rpcz_store.cc:276] 0709 13:11:09.025076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.037084 (+ 12008us) service_pool.cc:224] Handling call
0709 13:11:09.234483 (+197399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:09.326639 (+ 92156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:10.499157 (+1172518us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400014850. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:10.506933 (+  7776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238517632}
I20260709 13:11:10.558972 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:10.547335 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:10.560515 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:10.517907 29709 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:11:10.561945 30669 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:10.562357 30669 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:10.562800 30396 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:10.524885 30391 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.558264 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:10.551162 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:10.564142 29080 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:10.518352 30830 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.570007 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:10.571132 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:10.537525 30798 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.540973 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:10.588753 29080 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:10.530154 30733 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:11:10.564579 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:10.569038 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:10.563192 30669 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.571694 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.586268 30798 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:10.585872 29077 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:10.572451 30830 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:10.592177 30479 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:11:10.614250 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 138592us
I20260709 13:11:10.546252 30808 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.616317 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:10.620810 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:10.412470 30823 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:10.610909 30834 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:11:10.624887 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:10.494764 30793 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.617165 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.617666 30669 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:10.497232 30805 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:10.638638 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 84772.6us
I20260709 13:11:10.643059 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:10.566655 30827 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.619674 30799 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:10.624233 30801 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:10.642410 29154 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:10.639381 29711 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:10.597831 30735 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.660583 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:10.642858 30804 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.652562 30651 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:10.670034 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:10.657815 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:10.649266 30842 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:10.664865 30814 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.667126 30827 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:10.668336 30826 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:11:10.662797 30812 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:10.705878 29050 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.952s (SENT) [suppressed 31 similar messages]
I20260709 13:11:10.662911 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 137) took 1008 ms. Trace:
I20260709 13:11:10.651311 30789 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.653887 30801 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:10.683166 30823 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:09.838166 30605 consensus_queue.cc:1175] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "243d4ae4a4ae48b2979e32874472d256" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:11:10.683521 30809 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.481s	user 0.002s	sys 0.000s
I20260709 13:11:10.685873 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:10.684659 30804 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:10.689143 30267 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:11:10.689440 30842 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:10.702342 30799 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:10.702827 30806 consensus_meta.cc:287] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.008s
I20260709 13:11:10.726512 30806 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:10.703070 30805 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:10.703321 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.703589 30836 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.727540 30836 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.728153 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:10.729923 30836 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.703987 30827 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.713863 30801 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:10.691629 30735 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:10.704772 30812 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:10.723882 30817 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:11:10.724836 30842 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.725721 30799 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:10.663328 30762 consensus_queue.cc:1175] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "befda4a6c8344c26b0f139e4e1686b3b" responder_term: 3 status { last_received { term: 3 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 6 } }
I20260709 13:11:10.717532 29076 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:10.838095 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:10.708138 29154 rpcz_store.cc:276] 0709 13:11:09.654126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.517710 (+863584us) service_pool.cc:224] Handling call
0709 13:11:10.661690 (+143980us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:10.719561 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:10.840755 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 134895us
I20260709 13:11:10.742938 30391 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:10.855968 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:10.866107 29077 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
W20260709 13:11:10.866387 30763 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.701s	user 0.009s	sys 0.000s
I20260709 13:11:10.867003 30763 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.867847 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:10.845047 30504 consensus_meta.cc:287] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.147s	user 0.012s	sys 0.000s
I20260709 13:11:10.689819 30814 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.871234 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:10.845747 30844 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:11:10.838758 30812 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:10.715939 30789 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:10.880523 30805 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:10.885141 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:10.911954 29083 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
W20260709 13:11:10.888960 30843 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:11:10.891206 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:10.896438 30825 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.802557 30827 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.902223 30799 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:10.929723 29084 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:10.902850 30577 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.903353 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:10.935748 30637 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:10.904696 30838 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:11:10.905294 30651 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:10.913417 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:10.910843 30761 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.918259 29006 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:11:10.876339 30659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.383s	user 0.031s	sys 0.063s
I20260709 13:11:10.924523 30678 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:10.877549 30391 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:10.936419 30637 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.927973 30798 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.932631 30805 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:10.950582 30659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.461s	user 0.032s	sys 0.064s
I20260709 13:11:10.962680 29080 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:10.939009 29709 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:10.950505 30730 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:10.953243 30678 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:10.954519 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1041) took 1302 ms. Trace:
I20260709 13:11:10.941269 30577 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:10.966969 30671 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.906s	user 0.019s	sys 0.000s
I20260709 13:11:10.976954 29001 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:10.957763 30391 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.977593 30678 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.970925 29154 rpcz_store.cc:276] 0709 13:11:09.652242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.890865 (+1238623us) service_pool.cc:224] Handling call
0709 13:11:10.951682 (+ 60817us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:10.978971 30678 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:10.976853 30763 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.979717 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:10.980937 30637 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:10.978281 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1400) took 2005 ms. Trace:
I20260709 13:11:10.957985 30669 consensus_queue.cc:1059] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 13:11:10.983376 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:10.983141 29001 rpcz_store.cc:276] 0709 13:11:08.972446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.978548 (+  6102us) service_pool.cc:224] Handling call
0709 13:11:08.978795 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:10.977328 (+1998533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:10.978056 (+   728us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54001860d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:10.978217 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4594394496}
I20260709 13:11:10.965534 30659 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:10.990140 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:10.971729 30786 consensus_meta.cc:287] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.819s	user 0.012s	sys 0.016s
I20260709 13:11:10.953635 30761 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.993820 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:10.973338 30718 consensus_meta.cc:287] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.773s	user 0.014s	sys 0.053s
I20260709 13:11:10.970156 30730 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:10.978929 30391 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.969489 30793 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:10.938488 30825 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.995664 29001 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:10.980962 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:10.982625 30829 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.002050 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:10.967996 30830 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:10.998633 30730 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:11.001864 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.010628 30682 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:11.011057 30682 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:11.011152 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:11.011739 29085 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:11.012729 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:11.013688 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:11.014321 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.004161 30805 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.011865 30682 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.017095 30659 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.019181 29001 raft_consensus.cc:2393] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:11.000404 29155 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:11.020139 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1032) took 1593 ms. Trace:
I20260709 13:11:11.020254 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.022590 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:11.023862 29001 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:11.023015 29156 rpcz_store.cc:276] 0709 13:11:09.427205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.548169 (+120964us) service_pool.cc:224] Handling call
0709 13:11:11.019480 (+1471311us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:11.019670 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.019828 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.019849 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.019884 (+    35us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:11.019911 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.019958 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.020086 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30042496}
W20260709 13:11:11.009559 30780 consensus_meta.cc:287] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.003s	user 0.010s	sys 0.002s
I20260709 13:11:10.997473 30786 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.025807 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.006091 30799 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:11.005434 30829 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.033059 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.053692 29083 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:11.061748 29001 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:11.000242 30718 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.063482 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.063170 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1801) took 1256 ms. Trace:
I20260709 13:11:11.062228 30735 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.062299 30799 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:11.065369 29156 rpcz_store.cc:276] 0709 13:11:09.806856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.024746 (+1217890us) service_pool.cc:224] Handling call
0709 13:11:11.063091 (+ 38345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.065721 30799 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.065745 30735 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.067610 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:11.068193 29709 raft_consensus.cc:1813] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:11.068786 29709 raft_consensus.cc:2410] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:11:11.069229 30735 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.060556 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1047) took 1154 ms. Trace:
I20260709 13:11:11.069919 29155 rpcz_store.cc:276] 0709 13:11:09.885715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.727850 (+842135us) service_pool.cc:224] Handling call
0709 13:11:11.040443 (+312593us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.070873 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.062445 30825 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.027017 30827 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.026697 29005 raft_consensus.cc:2410] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:11.062638 30780 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:11.030087 30813 consensus_meta.cc:287] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.723s	user 0.014s	sys 0.002s
I20260709 13:11:11.071568 30812 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.023315 30391 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:11.030223 30577 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.090224 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:11.090570 29085 raft_consensus.cc:1813] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:11.094313 29085 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:11.098336 29154 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:11.077579 29081 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:11.102648 30669 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:11.102758 30853 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:11:11.079870 30799 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.081648 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1036) took 1549 ms. Trace:
I20260709 13:11:11.104401 29153 rpcz_store.cc:276] 0709 13:11:09.532571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.584237 (+ 51666us) service_pool.cc:224] Handling call
0709 13:11:11.072342 (+1488105us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:11.072524 (+   182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.081296 (+  8772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.081334 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.081385 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:11.081424 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.081490 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.081599 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8373,"raft.queue_time_us":8680,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":29470592}
I20260709 13:11:11.084937 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.106983 29002 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:11.107893 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.103119 30669 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.108935 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.109581 29002 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 2: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:11.110189 30669 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.110255 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.080226 30813 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.073545 30761 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:11.112489 30850 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:11:11.112521 29079 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:11.113026 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.087484 29709 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:11.113605 30810 consensus_meta.cc:287] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.000s
I20260709 13:11:11.073397 30830 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.114076 30810 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.086433 30801 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:11.073989 30769 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.532s	user 0.001s	sys 0.010s
I20260709 13:11:11.141227 30829 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.087822 30391 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.142711 30391 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:11.143618 30776 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.602s	user 0.001s	sys 0.014s
I20260709 13:11:11.114431 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.144376 29002 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:11.082996 29717 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:11.113637 29153 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:11.150494 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 133) took 1497 ms. Trace:
I20260709 13:11:11.150951 29153 rpcz_store.cc:276] 0709 13:11:09.653081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.112797 (+1459716us) service_pool.cc:224] Handling call
0709 13:11:11.150423 (+ 37626us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.152199 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.074656 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.159431 29156 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:11.159523 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:11.163020 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:11.029206 30789 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.028304 30808 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.072000 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.194458 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.104099 30577 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:11.183363 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.201210 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.198377 30582 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:11:11.183139 30793 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:11.202875 30778 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:11.094614 30735 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.202775 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:11.205943 29682 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:11.087546 30812 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:11.203868 30856 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 13:11:11.210702 29153 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:11.204660 30577 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:11.207643 30504 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.205669 30733 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:11.215312 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.233686 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.235121 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.212997 30798 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.207892 30841 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.221768 30577 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:11.245849 29156 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:11.226323 30793 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:11.212935 30735 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.235924 29155 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:11.253297 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:11.231178 30733 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.230561 30504 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:11.262516 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 57422us
I20260709 13:11:11.265931 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1800) took 1863 ms. Trace:
I20260709 13:11:11.266347 29151 rpcz_store.cc:276] 0709 13:11:09.402038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.402203 (+   165us) service_pool.cc:224] Handling call
0709 13:11:09.543874 (+141671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:09.546703 (+  2829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.265659 (+1718956us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002a35d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:11.265856 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325144832}
I20260709 13:11:11.274452 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.275503 29151 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:11.254590 30793 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:11.280609 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.282100 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.282917 29151 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:11.285127 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:11.286842 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 80963.4us
I20260709 13:11:11.245538 30841 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.287919 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 132) took 1635 ms. Trace:
I20260709 13:11:11.291440 29151 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:11.287740 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:11.291504 29156 rpcz_store.cc:276] 0709 13:11:09.652795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.287030 (+1634235us) service_pool.cc:224] Handling call
0709 13:11:11.287561 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.287699 (+   138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.287716 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.287749 (+    33us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:11.287767 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.287800 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.287881 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.293696 29682 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:11.292361 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1042) took 1639 ms. Trace:
I20260709 13:11:11.295145 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 138) took 1640 ms. Trace:
I20260709 13:11:11.294850 29151 rpcz_store.cc:276] 0709 13:11:09.652521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.288475 (+1635954us) service_pool.cc:224] Handling call
0709 13:11:11.291131 (+  2656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.292101 (+   970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.292306 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.300741 30841 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:11.301350 30841 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.272861 30797 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:11:11.305222 30733 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.306061 30735 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.306768 30504 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.298326 29151 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:11.307927 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 140) took 1653 ms. Trace:
I20260709 13:11:11.291314 30718 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:11.287312 30793 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:11.308353 29151 rpcz_store.cc:276] 0709 13:11:09.654867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.297744 (+1642877us) service_pool.cc:224] Handling call
0709 13:11:11.298103 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.307579 (+  9476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.307847 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.310712 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 141) took 1645 ms. Trace:
I20260709 13:11:11.296135 29156 rpcz_store.cc:276] 0709 13:11:09.654384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.293813 (+1639429us) service_pool.cc:224] Handling call
0709 13:11:11.294493 (+   680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.294734 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.294772 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.294833 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:11.294876 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.294945 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.295078 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169344}
I20260709 13:11:11.313211 29156 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:11.313971 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 143) took 1591 ms. Trace:
I20260709 13:11:11.314510 29156 rpcz_store.cc:276] 0709 13:11:09.722944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.312474 (+1589530us) service_pool.cc:224] Handling call
0709 13:11:11.312841 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.313691 (+   850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.313920 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.311153 29151 rpcz_store.cc:276] 0709 13:11:09.665148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.309546 (+1644398us) service_pool.cc:224] Handling call
0709 13:11:11.310081 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.310293 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.310330 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.310387 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:11.310428 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.310496 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.310636 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.319766 30391 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.298240 30863 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.294317 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.323521 29155 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:11.324506 30391 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.319825 29151 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:11.323868 29001 raft_consensus.cc:2410] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:11.292965 30786 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.331504 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1045) took 1517 ms. Trace:
I20260709 13:11:11.332217 29151 rpcz_store.cc:276] 0709 13:11:09.813920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.319100 (+1505180us) service_pool.cc:224] Handling call
0709 13:11:11.319499 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:11.331146 (+ 11647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.331429 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.347882 30730 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:11.349915 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 70818.6us
W20260709 13:11:11.351011 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.358561 30793 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.361966 30730 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.365309 30577 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.367751 29154 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:11.372399 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1043) took 1563 ms. Trace:
I20260709 13:11:11.372535 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1044) took 1563 ms. Trace:
I20260709 13:11:11.372848 29154 rpcz_store.cc:276] 0709 13:11:09.808856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.312971 (+1504115us) service_pool.cc:224] Handling call
0709 13:11:11.367521 (+ 54550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.372131 (+  4610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.372342 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.366699 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:11.367635 30792 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:11.373180 29156 rpcz_store.cc:276] 0709 13:11:09.809123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.317186 (+1508063us) service_pool.cc:224] Handling call
0709 13:11:11.317537 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.317814 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:11.318183 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 13:11:11.371977 (+ 53794us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.372061 (+    84us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:11.372242 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.372325 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.372467 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2172,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":97,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":38656,"thread_start_us":52984,"threads_started":1,"wal-append.queue_time_us":53452}
I20260709 13:11:11.377902 29156 raft_consensus.cc:3055] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:11.378409 29156 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:11.375105 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 81476us
I20260709 13:11:11.379410 29156 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.382108 29156 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:11.375648 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:11.385281 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:11.387420 29150 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:11.387169 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:11.388470 30584 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:11.388947 30584 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:11.389360 30584 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:11.375893 30792 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:11.392843 30792 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.393831 30792 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:11.387164 29158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1795) took 2219 ms (client timeout 1952 ms). Trace:
W20260709 13:11:11.394949 29158 rpcz_store.cc:269] 0709 13:11:09.167875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.168077 (+   202us) service_pool.cc:224] Handling call
0709 13:11:11.386957 (+2218880us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b5400055854. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:11.387101 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325583104}
I20260709 13:11:11.397836 29156 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:11:11.398240 29156 pending_rounds.cc:85] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 5
I20260709 13:11:11.398476 30582 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.380769 30805 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:11.394250 30733 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:11.398494 29156 pending_rounds.cc:107] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:11:11.400480 30577 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.400758 29156 raft_consensus.cc:2889] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:11.402202 30733 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.403307 30793 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.404471 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:11.408420 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:11.416019 30584 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.422874 30584 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:11.418519 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:11.425381 30813 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.425522 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:11.427713 29076 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:11.489600 29076 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:11.490345 30313 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:11.490805 30313 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:11.492846 30550 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:11.493279 30550 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:11.494138 30550 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.495182 30550 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:11.504504 29076 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:11.505998 29076 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:11.491742 30313 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.512255 30313 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.519791 30598 raft_consensus.cc:3055] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:11.519018 30794 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:11.520283 30598 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:11.521149 30598 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.522145 30598 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:11.532917 30730 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.533509 30730 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.535522 30730 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.538792 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.539405 29001 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:11.540316 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.542506 28975 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:11.543514 29001 raft_consensus.cc:2410] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:11.545931 29001 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:11:11.548566 29001 pending_rounds.cc:85] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 1
I20260709 13:11:11.548846 29001 pending_rounds.cc:107] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:11:11.549695 29001 raft_consensus.cc:2889] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:11.550436 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1425) took 1270 ms. Trace:
I20260709 13:11:11.550817 29001 rpcz_store.cc:276] 0709 13:11:10.279938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.544570 (+1264632us) service_pool.cc:224] Handling call
0709 13:11:11.545005 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.550196 (+  5191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.550384 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:11.555984 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 152) took 1007 ms. Trace:
I20260709 13:11:11.545691 28975 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:11.556450 29001 rpcz_store.cc:276] 0709 13:11:10.548675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.553776 (+1005101us) service_pool.cc:224] Handling call
0709 13:11:11.554062 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.554249 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.554279 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.554320 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:11.554353 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.554409 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.555926 (+  1517us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:11.569775 29122 leader_election.cc:336] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.633s (SENT)
W20260709 13:11:11.572576 29122 leader_election.cc:336] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.666s (SENT)
I20260709 13:11:11.576268 29001 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:11.591936 30814 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.598196 29001 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:11:11.603264 29159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1782) took 2672 ms (client timeout 1649 ms). Trace:
W20260709 13:11:11.603696 29159 rpcz_store.cc:269] 0709 13:11:08.930418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.930549 (+   131us) service_pool.cc:224] Handling call
0709 13:11:11.603079 (+2672530us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54002a3ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:11.603192 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3018883840}
I20260709 13:11:11.606278 30669 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.606755 30669 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.608385 30669 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.608773 30669 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.609565 30669 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.611965 30669 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.617765 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33750 (request call id 153) took 1068 ms. Trace:
I20260709 13:11:11.618355 29005 rpcz_store.cc:276] 0709 13:11:10.548947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.559854 (+ 10907us) service_pool.cc:224] Handling call
0709 13:11:11.617462 (+1057608us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54003f16d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:11.617681 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984126848}
I20260709 13:11:11.622138 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1028) took 2196 ms. Trace:
I20260709 13:11:11.622694 29157 rpcz_store.cc:276] 0709 13:11:09.426162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.523927 (+ 97765us) service_pool.cc:224] Handling call
0709 13:11:09.524256 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:09.721940 (+197684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.621783 (+1899843us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54002a30d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:11.622035 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446150016}
I20260709 13:11:11.627590 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1820) took 1662 ms. Trace:
I20260709 13:11:11.628124 29007 rpcz_store.cc:276] 0709 13:11:09.965354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.969392 (+  4038us) service_pool.cc:224] Handling call
0709 13:11:11.627370 (+1657978us) spinlock_profiling.cc:243] Waited 91.4 ms on lock 0x7b54000af0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:11.627516 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210206080}
I20260709 13:11:11.656425 30735 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.657480 30735 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.666224 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 937) took 1705 ms. Trace:
I20260709 13:11:11.666839 30751 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.666683 29082 rpcz_store.cc:276] 0709 13:11:09.961229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.964271 (+  3042us) service_pool.cc:224] Handling call
0709 13:11:09.964575 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:11.254253 (+1289678us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b180015f040. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f4115839060 00007f4115839bfb 00007f411584464b
0709 13:11:11.254384 (+   131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.254408 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:11.254837 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 13:11:11.665776 (+410939us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:11.665822 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.665909 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.665956 (+    47us) raft_consensus.cc:1685] finished
0709 13:11:11.665968 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.666162 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48701,"raft.queue_time_us":69544,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":13830,"spinlock_wait_cycles":114371968,"thread_start_us":454970,"threads_started":2,"wal-append.queue_time_us":25540}
W20260709 13:11:11.676963 28975 leader_election.cc:336] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.513s (SENT)
I20260709 13:11:11.677595 28975 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:11.680665 29050 leader_election.cc:336] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.570s (SENT)
I20260709 13:11:11.683749 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:11.683985 30839 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:11.684613 30814 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:11.690599 30783 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:11.692768 30814 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:11.693385 30814 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.696246 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:11.697311 30814 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.700843 30868 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.705375 30868 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.708518 30868 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.713021 30814 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:11.719178 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1372) took 1224 ms. Trace:
I20260709 13:11:11.720628 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:11.722348 30813 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:11.724423 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:11.731065 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:11.740980 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.731105 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 183) took 1587 ms. Trace:
I20260709 13:11:11.741710 30839 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.731575 30730 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:11.742522 30839 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.743209 30730 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.745220 30730 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.745362 30839 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.731770 30605 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.747131 30605 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.748746 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:11.749120 30605 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.719561 29078 rpcz_store.cc:276] 0709 13:11:10.495260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.936907 (+441647us) service_pool.cc:224] Handling call
0709 13:11:10.958812 (+ 21905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:10.997727 (+ 38915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:10.997773 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:10.997831 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:10.997876 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.718955 (+721079us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b5400199954. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:11.718999 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.719120 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89048960}
I20260709 13:11:11.751263 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.751359 29007 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:11.751832 29009 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:11.756477 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.757282 29007 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:11.733052 30813 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:11.758714 30813 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:11.759093 30813 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:11.759425 30813 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:11.759972 29050 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:11.763149 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1371) took 1279 ms. Trace:
I20260709 13:11:11.763931 30605 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.768337 30605 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.763521 29084 rpcz_store.cc:276] 0709 13:11:10.485626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.935489 (+449863us) service_pool.cc:224] Handling call
0709 13:11:10.935759 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:10.944608 (+  8849us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:10.957815 (+ 13207us) log.cc:844] Serialized 14 byte log entry
0709 13:11:11.002081 (+ 44266us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.002130 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:11.002173 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.030780 (+ 28607us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b18003fa720. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:11:11.088711 (+ 57931us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b480015c258. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411a76aaa1 00007f411a761b37 00007f411a7619ca 00007f411a85926f 00007f411a858f37 00007f411a86d55c 00007f411a86d508 00007f411a86d488 00007f411a86d443 00007f411a86c6ec 00007f4119796a96
0709 13:11:11.762927 (+674216us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b5400022954. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:11.762977 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.763095 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6015,"prepare.queue_time_us":13292,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":3261,"raft.run_cpu_time_us":2153,"raft.run_wall_time_us":84690,"spinlock_wait_cycles":40375936,"thread_start_us":41632,"threads_started":1,"wal-append.queue_time_us":52486}
I20260709 13:11:11.769479 30801 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.770063 30801 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.740432 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:11.770674 30801 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.741849 29079 rpcz_store.cc:276] 0709 13:11:10.143991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.265484 (+121493us) service_pool.cc:224] Handling call
0709 13:11:10.265794 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.113146 (+847352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.730893 (+617747us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b5400402254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:11.731050 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1946795264}
I20260709 13:11:11.771317 30801 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:11.737742 29708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 273) took 3552 ms (client timeout 1505 ms). Trace:
I20260709 13:11:11.771672 30801 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.771852 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:11.772138 30801 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:11.772562 30585 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 13:11:11.771896 29708 rpcz_store.cc:269] 0709 13:11:08.185728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.188249 (+  2521us) service_pool.cc:224] Handling call
0709 13:11:11.737487 (+3549238us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400577bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:11.737654 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4157365504}
I20260709 13:11:11.773115 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:11.775388 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.781106 30839 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.781772 30839 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.785401 30839 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.780264 30605 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.787564 30801 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.788297 29009 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:11.789551 30801 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:11.794899 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:11.798921 29152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1790) took 2763 ms (client timeout 3243 ms). Trace:
W20260709 13:11:11.799319 29152 rpcz_store.cc:269] 0709 13:11:09.036084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.036233 (+   149us) service_pool.cc:224] Handling call
0709 13:11:11.798737 (+2762504us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54003f66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:11.798868 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950665088}
I20260709 13:11:11.803123 30762 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.803217 29078 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:11.803674 30762 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.805716 29078 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:11.806067 30825 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:11.806473 30825 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.806829 30825 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:11.807121 30814 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:11.812318 30659 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.812460 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:11.812839 30659 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.813526 29078 raft_consensus.cc:2368] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
W20260709 13:11:11.818077 29048 leader_election.cc:336] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.733s (SENT)
I20260709 13:11:11.818665 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:11.819149 29002 raft_consensus.cc:2393] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:11.815824 30605 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.821710 30814 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.821475 30605 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.821009 29078 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:11:11.824023 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.825981 29008 raft_consensus.cc:2410] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:11.826412 30669 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.826879 30669 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.829805 30813 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.827256 30825 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.834337 30825 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.834960 30825 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:11.835297 30825 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:11.835669 30825 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:11.836853 30813 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.839748 30849 raft_consensus.cc:3055] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:11.840166 30849 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:11.842024 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.842646 29009 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:11.848021 30799 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.848654 30799 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:11.850190 29009 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:11.852948 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1369) took 1557 ms. Trace:
I20260709 13:11:11.853564 29077 rpcz_store.cc:276] 0709 13:11:10.295728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.937108 (+641380us) service_pool.cc:224] Handling call
0709 13:11:10.971683 (+ 34575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:11.003143 (+ 31460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.004606 (+  1463us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:11.088214 (+ 83608us) log.cc:844] Serialized 26 byte log entry
0709 13:11:11.285577 (+197363us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:11.285629 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.285714 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.852717 (+567003us) raft_consensus.cc:1685] finished
0709 13:11:11.852736 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.852885 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1089,"raft.queue_time_us":3895,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":4345,"thread_start_us":69674,"threads_started":1,"wal-append.queue_time_us":186041}
I20260709 13:11:11.854946 30825 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.856437 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.857095 29005 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:11.849771 30849 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.858566 30849 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.852933 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:11.865420 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.865942 30798 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:11.866422 29009 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:11.866585 30798 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:11.864742 29005 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:11.867524 30798 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.868660 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.869253 29009 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.880911 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:11.881659 30825 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.882551 30730 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:11.884773 29009 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:11.885272 30825 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:11.886092 29008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1819) took 2040 ms (client timeout 1733 ms). Trace:
I20260709 13:11:11.886468 29004 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:11.886193 30825 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:11.886557 29008 rpcz_store.cc:269] 0709 13:11:09.846066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.849722 (+  3656us) service_pool.cc:224] Handling call
0709 13:11:11.825582 (+1975860us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b54001499a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157b8dc0 00007f4115844e18 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:11.885898 (+ 60316us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54003ab454. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:11.886031 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1926551680}
I20260709 13:11:11.888540 30825 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.889276 30811 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:11.892308 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.892645 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.889653 29051 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:11.893815 30605 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.894655 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:11.894723 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.895004 30811 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:11.895534 29005 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:11.895833 29001 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:11.896948 29009 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:11.897575 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1424) took 1617 ms. Trace:
I20260709 13:11:11.897930 29006 rpcz_store.cc:276] 0709 13:11:10.279701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.653991 (+374290us) service_pool.cc:224] Handling call
0709 13:11:10.664178 (+ 10187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:10.944720 (+280542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.897408 (+952688us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b5400291b54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:11.897533 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583269248}
I20260709 13:11:11.898456 30868 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:11.899099 29680 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:11.899901 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.900146 30868 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:11.900552 30868 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:11.900894 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:11.901568 29007 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:11.901005 30783 consensus_queue.cc:1059] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:11.902767 30783 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:11.903424 30783 raft_consensus.cc:3055] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:11.903649 30783 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:11.903717 30811 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:11.904386 30811 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.904353 30783 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.905275 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:11.900918 30868 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.949100 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 934) took 1993 ms. Trace:
I20260709 13:11:11.915879 29680 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:11.942287 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1365) took 1647 ms. Trace:
I20260709 13:11:11.955996 29081 rpcz_store.cc:276] 0709 13:11:09.955434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.955899 (+   465us) service_pool.cc:224] Handling call
0709 13:11:09.956163 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.102895 (+1146732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.948767 (+845872us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400146c54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:11.949020 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2577991552}
I20260709 13:11:11.906525 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:11.957821 29083 rpcz_store.cc:276] 0709 13:11:10.294602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.912808 (+618206us) service_pool.cc:224] Handling call
0709 13:11:10.913169 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.062951 (+149782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.941914 (+878963us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000c80d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:11.942199 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322547456}
I20260709 13:11:11.901619 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.957180 30793 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:11.936496 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.959479 30793 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:11.959988 30793 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:11.907213 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:11.952850 30811 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.959410 30783 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.966570 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:11.967284 30810 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.967814 30811 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:11.968603 30577 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:11.968626 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:11.969389 29711 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:11.969233 30577 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.970286 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:11.971326 30577 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.971747 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:11.972358 29083 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:11.973635 30716 consensus_queue.cc:1059] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:11.974602 30716 raft_consensus.cc:3055] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:11.974943 30716 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:11.974798 30868 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.976248 29708 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:11.976085 30716 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:11.977241 30716 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.977540 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:11.977865 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:11.978848 30780 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:11.979333 30780 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:11.968405 30811 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:11.980211 30780 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:11.981926 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:11.978221 29077 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.982442 29083 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:11.982501 29078 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:11.983582 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:11.984831 30794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35252 (request call id 74) took 5383 ms. Trace:
I20260709 13:11:11.985198 30794 rpcz_store.cc:276] 0709 13:11:06.615469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.620691 (+  5222us) service_pool.cc:224] Handling call
0709 13:11:06.736247 (+115556us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b54001cdf20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157be520 00007f4115847b42 00007f4115846800 00007f411583f9fa 00007f411a8074c9 00007f4113d83c54 00007f4113d83be2 00007f4113d83b11 00007f4113d83a8c 00007f4113d82d42
0709 13:11:06.855807 (+119560us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b58003899d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411579d0aa 00007f411581f447 00007f411583c70a 00007f411583d76a 00007f411583c91b 00007f411584803c 00007f41158476b3 00007f411583f9fa 00007f411a8074c9 00007f4113d83c54
0709 13:11:06.943084 (+ 87277us) log.cc:844] Serialized 494 byte log entry
0709 13:11:08.633628 (+1690544us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b18003d60c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f411579ab0b 00007f411581fb63 00007f4115847752 00007f411583f9fa
0709 13:11:08.663811 (+ 30183us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b18003d60c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:11:11.984744 (+3320933us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":949834,"raft.queue_time_us":961022,"raft.run_cpu_time_us":925,"raft.run_wall_time_us":10004,"spinlock_wait_cycles":287280256,"thread_start_us":698696,"threads_started":1,"wal-append.queue_time_us":972223,"wal-append.run_cpu_time_us":2339,"wal-append.run_wall_time_us":1096477}
W20260709 13:11:11.990108 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.987490 30811 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:11.990898 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.994815 30868 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:11.995662 30659 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:11.997009 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1810) took 2267 ms. Trace:
I20260709 13:11:11.997398 29010 rpcz_store.cc:276] 0709 13:11:09.729786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.732348 (+  2562us) service_pool.cc:224] Handling call
0709 13:11:11.916653 (+2184305us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:11.916834 (+   181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:11.917148 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.917176 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:11.917634 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:11:11.980951 (+ 63317us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:11.981248 (+   297us) log.cc:844] Serialized 14 byte log entry
0709 13:11:11.981721 (+   473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.981788 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.996692 (+ 14904us) raft_consensus.cc:1685] finished
0709 13:11:11.996791 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.996930 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":152,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":4190,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":232,"spinlock_wait_cycles":21013248,"thread_start_us":62982,"threads_started":1,"wal-append.queue_time_us":63311}
I20260709 13:11:11.998675 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:11.999451 29003 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:11.997299 29711 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
W20260709 13:11:12.000525 29122 leader_election.cc:336] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.766s (SENT)
W20260709 13:11:12.001367 29122 leader_election.cc:336] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.803s (SENT)
W20260709 13:11:12.002890 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.003019 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1423) took 1724 ms. Trace:
I20260709 13:11:12.003446 29004 rpcz_store.cc:276] 0709 13:11:10.279174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.626771 (+347597us) service_pool.cc:224] Handling call
0709 13:11:11.886151 (+1259380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.886992 (+   841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.002731 (+115739us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003ab450. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:12.002951 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2843220736}
W20260709 13:11:12.008214 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:12.008654 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.004866 30659 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.009935 30659 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.010324 30391 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:12.010885 30391 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:12.013110 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.013485 30391 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:12.015782 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1039) took 2372 ms. Trace:
I20260709 13:11:11.996502 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.016333 29150 rpcz_store.cc:276] 0709 13:11:09.643876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.644013 (+   137us) service_pool.cc:224] Handling call
0709 13:11:11.387242 (+1743229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.387837 (+   595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.015584 (+627747us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b5400055850. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:12.015713 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4008000384}
I20260709 13:11:12.017095 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.018469 29682 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.019189 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:12.019507 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:12.020476 30783 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.020959 29003 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:12.021176 30783 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:12.021667 30659 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:12.022214 30659 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:12.023324 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:12.025211 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:12.026328 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:12.026981 29713 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:12.027670 30504 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:12.028242 30504 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:12.028633 30504 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:12.029016 29051 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.030318 28905 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 2 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:12.033565 29680 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:12.035904 30808 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.036634 30504 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.037353 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.038141 29010 raft_consensus.cc:2410] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:11:12.038941 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.039012 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.039152 29711 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:12.039611 29007 raft_consensus.cc:2376] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4. Re-sending same reply.
I20260709 13:11:12.037973 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.041505 30868 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:12.042315 29010 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:12.042730 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.043411 29051 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:12.043926 29007 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:12.044225 30780 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:12.044493 29003 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:12.044623 30780 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:12.045029 30780 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:12.047308 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:12.047793 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.047576 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.049683 30868 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:12.050269 30868 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:12.051172 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:12.052016 30889 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:12.054364 29084 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:12.055529 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.055289 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.056147 29079 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:12.056486 29081 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:12.057044 30797 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:12.057453 30797 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:12.057562 29079 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:12.058288 30797 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.058925 29008 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.059612 30797 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:12.060561 30868 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.023550 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:12.041551 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.065876 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.067463 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.068219 30761 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:12.068702 30761 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:12.069628 30762 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:12.069550 30761 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.071168 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:12.025278 29083 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:12.072273 30811 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:12.072889 30504 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:12.073572 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2ed915f0be134d71b36d8fc8f7b5830b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:12.074076 30811 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:12.027640 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:12.063768 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:11.907013 30778 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:12.096681 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.097071 29711 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:12.097347 29006 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:12.074227 29714 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:12.098948 30762 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:12.100008 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.100837 29006 raft_consensus.cc:2410] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:12.102428 29077 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:12.103446 30682 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:12.101830 28905 catalog_manager.cc:5697] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 2 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.075484 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1362) took 1781 ms. Trace:
I20260709 13:11:12.132445 29080 rpcz_store.cc:276] 0709 13:11:10.293712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.589706 (+295994us) service_pool.cc:224] Handling call
0709 13:11:10.589949 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:10.964548 (+374599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.075273 (+1110725us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400404a54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:12.075425 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856594816}
I20260709 13:11:12.133673 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.135219 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:12.135871 29080 raft_consensus.cc:2368] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:12.075996 29716 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.140697 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.141418 29080 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:12.142720 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.143287 29080 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:12.144073 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.148170 29080 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:12.150800 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:12.076421 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:12.148479 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.152591 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.153877 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.154395 29156 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:12.078329 29710 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:12.149600 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.157353 29715 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.157748 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.158730 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.159631 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.078624 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.160888 29079 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:12.162060 29079 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:12.162420 29008 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:12.163062 29010 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.078845 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:12.079572 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.164587 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.165275 29010 raft_consensus.cc:2368] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:12.165449 29081 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:12.166023 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.166603 29081 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.166971 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.080598 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.169018 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.169170 29010 raft_consensus.cc:2368] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:12.086198 30585 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:11:12.170089 30585 consensus_queue.cc:1243] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:11:12.170292 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.170410 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.086278 29004 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:12.171346 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.171954 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.172196 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.173039 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.173269 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.173913 29010 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:12.174136 29004 raft_consensus.cc:2410] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:12.174702 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.174917 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.175257 29010 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:12.079861 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.176290 29010 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.176611 29084 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 1.
I20260709 13:11:12.082212 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.177352 29010 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:12.177745 29082 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:12.082473 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.178972 29085 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:12.086745 29007 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:12.181320 29007 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:12.086643 30479 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:11:12.085764 30637 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 13:11:12.091481 28906 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52674 (request call id 258) took 8027 ms. Trace:
I20260709 13:11:12.094600 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.193799 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.095212 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.196990 29152 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:12.198241 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.199589 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.095420 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.218106 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.219251 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.219542 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.095718 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.220621 30789 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.220757 29159 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:12.221987 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.223400 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.200591 30762 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:12.224556 30762 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:12.225015 30762 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:12.226070 30762 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.227588 30762 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:12.095839 29003 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:12.096112 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.229460 29002 raft_consensus.cc:2368] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:12.096393 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.231307 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.088027 30718 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.234148 30718 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.236953 30718 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:12.074790 30811 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:12.238580 29151 raft_consensus.cc:2410] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:12.240676 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.242288 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.105631 29006 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.106261 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.245137 29009 raft_consensus.cc:2368] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:12.107208 28905 catalog_manager.cc:5697] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 2 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 3 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:12.248003 28905 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 2, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:12.249890 28905 catalog_manager.cc:5697] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 3, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:12.107651 29001 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:11:12.108502 29005 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:12.085258 30706 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.122s
I20260709 13:11:12.111622 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:12.074934 29713 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:12.113313 29122 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.120311 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:12.260756 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 140515us
W20260709 13:11:12.261051 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 147797us
I20260709 13:11:12.261674 30761 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:12.262171 30761 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:12.091830 30780 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.263079 30761 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.263592 30868 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:12.263974 29076 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.264087 30780 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:12.209087 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.105332 30812 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:12.265205 30812 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:12.266078 30812 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.267596 30812 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:12.269627 29005 pending_rounds.cc:85] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 1
I20260709 13:11:12.269963 29005 pending_rounds.cc:107] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:11:12.270229 29005 raft_consensus.cc:2889] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:12.264178 30868 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:12.273064 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.273411 30868 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.112704 29155 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:12.276226 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.277671 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.278380 29155 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:12.279428 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.153893 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:12.280473 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 126625us
I20260709 13:11:12.274220 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.167265 29683 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:12.282039 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 114849us
W20260709 13:11:12.282719 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.081918 30669 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:12.284243 30829 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:12.284602 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.285187 30669 raft_consensus.cc:1064] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:11:12.285204 29076 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:12.286412 28975 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:12.284248 30888 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:11:12.287405 30675 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:12.287814 30675 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.288196 30675 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.120625 29051 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:12.289335 30504 consensus_queue.cc:1059] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:11:12.290446 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.291339 30761 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:12.291831 30761 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:12.292464 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:12.292701 30761 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.293171 30504 consensus_queue.cc:1059] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:12.294551 30504 raft_consensus.cc:3055] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:12.294881 30504 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:12.289784 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 169246us
I20260709 13:11:12.291360 30797 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:12.295760 30504 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.297338 30504 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.216398 29085 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.131089 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.300215 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.301183 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.301352 30675 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.302134 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.293143 29006 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:12.282562 30659 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:12.284003 30783 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:12.304209 30783 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.304589 30783 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:12.169167 29078 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:12.243322 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.305414 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.306434 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.306697 30577 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:12.306960 29157 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:11:12.307282 30577 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:12.288972 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.307817 30577 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:12.303128 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.308336 29076 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:12.303834 30659 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.309074 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.309305 30659 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:12.309629 29154 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:12.310400 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.311317 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.296418 30797 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:12.312714 30797 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.094025 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.313973 30797 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:12.314455 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.108843 30786 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:11:12.316574 28975 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.316798 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.317718 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.318176 28975 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.318622 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.316608 30813 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:11:12.319315 29158 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:12.288887 30669 consensus_queue.cc:237] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.321389 30391 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.322078 30391 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:12.322845 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:12.324249 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.325306 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.325917 29154 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:12.326532 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.327404 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.328991 30659 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.329563 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:12.333338 29715 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:12.319984 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.334136 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:12.335893 29001 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:12.341384 29158 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.336432 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.338232 30783 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.188589 28906 rpcz_store.cc:276] 0709 13:11:04.064262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:04.080555 (+ 16293us) service_pool.cc:224] Handling call
0709 13:11:12.091419 (+8010864us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:06.790773 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:06.791339 (+   566us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:06.791369 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:11:06.791386 (+    17us) tablet.cc:662] Decoding operations
0709 13:11:06.795681 (+  4295us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:06.795749 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:06.795788 (+    39us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:11:06.796798 (+  1010us) tablet.cc:701] Row locks acquired
0709 13:11:06.796828 (+    30us) write_op.cc:260] PREPARE: finished
0709 13:11:06.797014 (+   186us) write_op.cc:270] Start()
0709 13:11:06.797140 (+   126us) write_op.cc:276] Timestamp: P: 1783602666796961 usec, L: 0
0709 13:11:06.799729 (+  2589us) op_driver.cc:348] REPLICATION: starting
0709 13:11:06.884629 (+ 84900us) log.cc:844] Serialized 3632 byte log entry
0709 13:11:08.268023 (+1383394us) op_driver.cc:464] REPLICATION: finished
0709 13:11:08.578376 (+310353us) write_op.cc:301] APPLY: starting
0709 13:11:08.584639 (+  6263us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:08.596260 (+ 11621us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:08.596384 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:08.597389 (+  1005us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:08.597601 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:11:08.597644 (+    43us) write_op.cc:312] APPLY: finished
0709 13:11:08.597895 (+   251us) log.cc:844] Serialized 71 byte log entry
0709 13:11:08.598059 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:08.598177 (+   118us) write_op.cc:454] Released schema lock
0709 13:11:08.598218 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:08.598579 (+   361us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:08.598611 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":270098,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":297956,"apply.run_cpu_time_us":2769,"apply.run_wall_time_us":22511,"num_ops":9,"prepare.queue_time_us":147961,"prepare.run_cpu_time_us":9507,"prepare.run_wall_time_us":736057,"raft.queue_time_us":665703,"raft.run_cpu_time_us":33386,"raft.run_wall_time_us":268798,"replication_time_us":1410065,"thread_start_us":1479437,"threads_started":4,"wal-append.queue_time_us":524148,"wal-append.run_cpu_time_us":40940,"wal-append.run_wall_time_us":2204206}]]}
I20260709 13:11:12.344975 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.345994 30659 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:12.346308 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.346963 29154 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:12.350656 30783 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:12.349198 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:12.352484 29076 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:12.359582 29157 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:12.360540 30675 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:12.361769 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.363341 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.364740 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.365507 29159 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:12.366542 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.367301 29159 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:12.369828 30479 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:12.370684 30479 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:11:12.372138 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 105485us
I20260709 13:11:12.374042 30637 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:11:12.375748 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.377101 29157 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:12.378230 29157 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:12.213526 29081 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.315830 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:12.289354 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:12.394567 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.312880 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.396531 29076 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:12.399734 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:12.350271 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:12.405174 29154 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:12.405869 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 93092.8us
I20260709 13:11:12.406433 29157 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:11:12.411593 29122 leader_election.cc:400] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:12.411975 29122 leader_election.cc:403] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:12.412737 29122 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:12.413471 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.413897 29122 leader_election.cc:400] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:12.414412 29122 leader_election.cc:403] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:11:12.416728 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.088850 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.409879 29158 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:12.347079 29152 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:12.427381 29713 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:12.418202 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.429615 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.430137 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:12.431146 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 342328us
I20260709 13:11:12.149101 29716 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
W20260709 13:11:12.447530 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.108403 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:12.453178 29080 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:12.454943 29051 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:12.096328 30778 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:12.456938 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 57267.2us
I20260709 13:11:12.432673 30706 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.462317 30808 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:12.463127 30808 raft_consensus.cc:697] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:12.463551 29711 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:12.464257 30808 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.467103 30577 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.435868 30585 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.468858 30678 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:12.469269 30678 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.469301 30577 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:12.450307 30595 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:11:12.470263 30605 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:11:12.470393 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52255.1us
I20260709 13:11:12.472123 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:12.469643 30678 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.476330 30789 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:12.476737 30789 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.477077 30789 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.078193 30793 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.482291 29717 raft_consensus.cc:2393] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:12.483866 30793 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:12.484522 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:12.496659 28975 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.104007 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:12.444597 29683 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:12.500747 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:12.500980 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56491.5us
I20260709 13:11:12.501508 29159 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:12.432771 30479 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:12.437476 29153 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:12.493171 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.499181 29156 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:12.500018 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.510254 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.510372 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.423120 30761 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:11:12.486707 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 56642us
I20260709 13:11:12.512295 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.504143 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:12.505440 29004 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:12.437747 30637 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.527166 29125 leader_election.cc:400] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:12.518878 29154 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:12.514521 29080 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:12.516592 30595 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:11:12.501327 30659 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:12.541934 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80704.6us
I20260709 13:11:12.542517 30659 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.498865 30810 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.544531 29051 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:12.544493 29083 raft_consensus.cc:2393] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:12.545360 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:12.545604 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:12.438555 30786 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:11:12.546244 29716 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:12.501801 30730 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:12.546211 29001 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:12.546602 29106 debug-util.cc:398] Leaking SignalData structure 0x7b08000e7e80 after lost signal to thread 27904
I20260709 13:11:12.548269 30794 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:12.578262 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.517627 30675 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:12.517717 30783 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:12.493741 30868 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 13:11:12.584151 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 72080.5us
I20260709 13:11:12.506963 30335 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:12.518335 30605 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:12.528023 30391 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:12.518180 30829 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:12.589670 29106 debug-util.cc:398] Leaking SignalData structure 0x7b08001211c0 after lost signal to thread 28942
W20260709 13:11:12.591182 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 514773us
I20260709 13:11:12.592638 30730 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:12.601725 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71972.6us
I20260709 13:11:12.604151 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.585965 30813 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:12.628222 29682 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:12.631222 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.545509 30789 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:12.594919 30891 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:11:12.592898 30479 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.657462 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:12.612411 30605 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.664165 29683 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.616022 30783 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:12.603052 30504 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:12.646138 29122 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.586861 30659 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:11:12.584019 30810 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.654206 29125 leader_election.cc:400] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:12.657723 29050 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.617844 30595 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:11:12.657119 30697 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.611341 29001 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:12.631620 30730 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.643376 30889 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:12.697705 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43642.5us
I20260709 13:11:12.698447 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.706991 29050 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.708858 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.710187 29050 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:12.623493 29080 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
W20260709 13:11:12.699278 29106 debug-util.cc:398] Leaking SignalData structure 0x7b0800392100 after lost signal to thread 29034
W20260709 13:11:12.711539 29106 debug-util.cc:398] Leaking SignalData structure 0x7b0800333e40 after lost signal to thread 29035
I20260709 13:11:12.699679 30808 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.679133 30504 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:12.662981 30813 raft_consensus.cc:697] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:12.666718 30605 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.684480 29711 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:12.712103 30761 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:12.723080 29050 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:12.723675 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 95582.2us
I20260709 13:11:12.676929 30335 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:11:12.701730 30718 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.662038 30678 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.676410 30682 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:11:12.716684 30811 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.726873 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.712838 30780 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.719003 30504 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:12.706665 30479 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:11:12.799499 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:12.771499 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 261306us
I20260709 13:11:12.765180 30793 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.802861 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:12.803889 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:12.771406 29106 debug-util.cc:398] Leaking SignalData structure 0x7b0800362360 after lost signal to thread 29109
I20260709 13:11:12.766654 30808 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.770643 30789 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:12.831149 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:12.840204 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.843824 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.848309 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:12.774443 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 47621.8us
I20260709 13:11:12.830346 30812 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:12.853086 30812 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:12.853596 30812 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:12.854444 30812 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:12.856781 30908 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
W20260709 13:11:12.857952 30912 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:11:12.829700 30780 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:12.841876 30839 raft_consensus.cc:2764] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:11:12.801118 30761 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:12.859207 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 343212us
I20260709 13:11:12.825620 30913 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:12.771134 30783 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:12.866771 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 202691us
I20260709 13:11:12.868993 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:12.842509 30762 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:11:12.880442 28975 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:12.882972 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.883104 29683 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:12.852191 30797 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:12.900334 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 198949us
I20260709 13:11:12.841104 30808 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.838989 30813 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:12.844867 30789 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 13:11:12.921096 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 72860.8us
I20260709 13:11:12.891697 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.883572 30913 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:12.927531 30913 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:12.927659 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.929601 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:12.934432 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.893735 30786 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:12.924680 30761 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:12.925788 29106 debug-util.cc:398] Leaking SignalData structure 0x7b08003ca740 after lost signal to thread 29183
W20260709 13:11:12.940436 29106 debug-util.cc:398] Leaking SignalData structure 0x7b08003cacc0 after lost signal to thread 29184
W20260709 13:11:12.941601 29106 debug-util.cc:398] Leaking SignalData structure 0x7b08003a0440 after lost signal to thread 29741
W20260709 13:11:12.941885 29106 debug-util.cc:398] Leaking SignalData structure 0x7b0800398de0 after lost signal to thread 29742
W20260709 13:11:12.944234 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 235968us
I20260709 13:11:12.944145 30914 consensus_queue.cc:1059] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:12.945286 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:12.922060 30839 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:12.892149 30915 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:12.951617 30762 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:12.965092 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 84740.2us
I20260709 13:11:12.968952 30936 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:12.970654 30914 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:12.939286 30810 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:12.938974 30786 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:12.951207 30889 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:12.975210 30930 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:12.956588 30812 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:12.946739 30922 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:11:12.983924 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 49581.8us
W20260709 13:11:12.986160 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56614.6us
I20260709 13:11:12.974929 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:12.980119 30901 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:11:12.974170 30915 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:12.981932 30904 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:11:12.977803 30678 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:12.991179 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45978.6us
I20260709 13:11:12.992264 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.979485 30930 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:12.976977 30786 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:12.961500 30585 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:11:12.986004 30889 raft_consensus.cc:697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:12.987181 30812 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:12.995798 30682 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:12.971344 30914 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.002830 30479 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:13.003268 30479 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:13.004087 30914 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:13.005035 30916 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:11:12.987573 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:12.990494 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65980.5us
I20260709 13:11:12.997907 29158 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:13.008221 29158 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:13.008340 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.984254 30335 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:13.009654 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:13.011646 29150 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:13.014407 29152 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:13.006353 30919 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:11:12.988437 30937 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:11:13.015920 30915 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.016086 30812 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:12.987315 30811 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:11:13.012557 29159 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:12.994566 30922 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:11:12.997038 30930 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:13.019459 29122 leader_election.cc:400] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:13.019806 29122 leader_election.cc:403] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:11:13.021250 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:13.021968 30924 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:13.024333 30911 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.253s	user 0.002s	sys 0.001s
I20260709 13:11:13.027832 30780 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:13.028964 30595 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.030198 29152 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:13.030332 30595 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:13.031366 30595 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:13.034399 30682 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:13.034777 30682 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:13.035101 30682 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:13.002955 30391 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:13.035535 30391 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:13.037741 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:13.039079 29004 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:13.039289 30926 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:13.039641 30926 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:12.996703 30927 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.040491 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:13.047441 29001 raft_consensus.cc:2368] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
W20260709 13:11:13.042418 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 50233.8us
I20260709 13:11:13.015241 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:13.014917 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:13.049427 29153 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:13.049611 29150 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:13.008973 29158 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.050871 29158 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:13.042122 30335 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:13.057091 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:13.001157 30931 raft_consensus.cc:2764] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:11:13.059028 28976 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:12.996014 30794 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:11:13.076165 29717 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:13.079021 29001 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:13.081220 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:13.088457 29003 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:13.042357 30789 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:13.015954 30585 raft_consensus.cc:3055] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:13.083741 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:13.067611 29153 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:13.085348 29005 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:13.040496 30926 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.051916 30550 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:13.095484 29006 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:13.104768 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:13.106542 29008 raft_consensus.cc:2410] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:11:13.052086 30792 consensus_queue.cc:1175] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "06ae4197fd784525b695a9b16c8094ea" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:11:13.042984 30595 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 13:11:13.109084 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65497.2us
W20260709 13:11:13.123718 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 74576.5us
I20260709 13:11:13.085814 30669 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:13.089821 30585 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:13.155332 29158 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:13.090662 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:13.069387 30839 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:11:13.004096 30479 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.058941 30927 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.073803 30678 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:11:13.069253 30718 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:11:13.086143 30933 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:13.058190 30930 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.178638 30930 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.098104 30789 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:13.179708 30789 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:13.182358 30812 raft_consensus.cc:3055] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.182835 30812 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:13.151815 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.183799 30812 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:13.184947 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.161309 30926 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:13.185477 30812 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:13.186549 30637 consensus_meta.cc:287] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.001s
I20260709 13:11:13.186931 30637 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:13.188787 30479 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:13.196117 30697 consensus_queue.cc:1175] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "befda4a6c8344c26b0f139e4e1686b3b" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:11:13.204404 30933 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:13.205689 30933 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.206470 30716 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.207010 30716 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.209187 30716 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:13.214974 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 130495us
W20260709 13:11:13.217355 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.218940 29158 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:13.219906 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:13.221177 29080 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:13.218997 29153 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:13.222527 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:13.221312 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:13.232895 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:13.240257 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:13.104817 30931 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
W20260709 13:11:13.244172 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.244073 29050 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:13.170204 29708 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:13.248921 29150 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:13.249265 29158 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:13.191747 30730 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:13.236496 30605 consensus_meta.cc:287] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.004s
I20260709 13:11:13.272029 29150 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:13.179868 30786 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.279546 29083 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:13.280911 30786 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.247987 30839 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:13.248489 30762 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:13.116447 30889 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.212154 30946 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:11:13.247985 30789 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.263324 30678 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.269681 30780 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:13.238083 30939 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:11:13.250243 30716 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:11:13.238401 30391 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.322327 29150 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:13.266688 30923 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:11:13.275337 30605 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.302938 30839 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:13.306330 30550 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:11:13.340499 30682 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.288393 30937 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:11:13.307940 30780 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:13.330677 30504 consensus_meta.cc:287] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
I20260709 13:11:13.307037 30678 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.380234 29159 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:11:13.356135 30944 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:11:13.356208 30780 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:13.392364 30940 log.cc:927] Time spent T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:11:13.334167 30913 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.386284 30948 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:13.358945 30504 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.392769 30391 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.418164 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:13.422612 29082 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:13.425115 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:13.407063 30924 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:13.450091 30945 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:13.450613 30945 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:13.450965 30945 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:13.452569 30955 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:11:13.453688 30951 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:13.478863 29716 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:13.568768 29003 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:13.600108 29003 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:13.531100 30829 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:13.608973 29003 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:13.544474 30930 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.545176 30730 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.544823 30950 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.616825 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:13.621785 29003 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:13.591322 30924 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:13.589221 30923 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:13.633637 29716 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:13.556201 30939 consensus_queue.cc:1059] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:11:13.410836 30948 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.608604 30829 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:13.426657 30839 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.606689 30789 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.609838 30797 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:13.654663 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:13.637140 29003 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.656994 30716 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
W20260709 13:11:13.657656 30953 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:11:13.660400 30931 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.660274 30585 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.661746 30335 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.664381 29003 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:13.635438 30605 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.613987 30927 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:13.633420 30930 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.689945 30930 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.637010 30950 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.694177 30950 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.655360 30682 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:13.695240 30913 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.697528 29711 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:11:13.699003 30812 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.003s
I20260709 13:11:13.702872 29153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
W20260709 13:11:13.674252 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 63812.4us
I20260709 13:11:13.671293 30829 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:13.676106 30335 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:13.677404 29708 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:13.656878 30924 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.684106 30958 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:13.683677 30937 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:13.684697 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.753592 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:13.655159 30923 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.797739 29152 raft_consensus.cc:2368] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:13.753877 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:13.799072 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46145.9us
I20260709 13:11:13.801386 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:13.801816 29158 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:13.802034 29152 raft_consensus.cc:2368] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:13.675323 30678 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.669780 30797 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.804750 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:13.806018 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 141855us
I20260709 13:11:13.806437 29152 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
I20260709 13:11:13.810444 29050 leader_election.cc:400] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:13.810750 29050 leader_election.cc:403] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:13.757308 30605 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:13.814661 29158 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:13.816814 29050 leader_election.cc:400] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:13.818512 29050 leader_election.cc:403] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:13.812300 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:13.799489 30829 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:13.844653 29050 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:13.823297 30813 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:13.816478 30504 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:13.760023 30793 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
W20260709 13:11:13.850108 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.824844 30678 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:13.810984 30972 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:13.822059 30605 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:13.817224 30945 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:13.869123 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:14.214478 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.230727 30504 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:13.861292 30972 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:14.386601 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.850584 30913 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:13.854696 30975 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.675149 30931 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.256072 30605 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:14.313710 30970 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.463s	user 0.007s	sys 0.007s
W20260709 13:11:14.314730 30973 log.cc:927] Time spent T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.459s	user 0.006s	sys 0.009s
I20260709 13:11:14.420565 30921 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:14.385610 30959 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.258898 30924 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:14.314026 30479 consensus_meta.cc:287] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.125s	user 0.016s	sys 0.008s
I20260709 13:11:14.261014 30923 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:14.411072 30974 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.550s	user 0.000s	sys 0.015s
I20260709 13:11:14.416970 30975 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.423502 30678 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:14.426678 30982 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:14.436367 30839 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.463117 30786 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:14.464520 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:14.464955 30780 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.070s	user 0.012s	sys 0.019s
W20260709 13:11:14.481492 30963 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:11:14.502995 30913 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:14.503150 30923 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:14.505451 30972 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.507992 30550 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:14.507476 30839 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.509063 30678 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.511312 30797 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.511485 29153 raft_consensus.cc:606] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:14.513909 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1393) took 1288 ms. Trace:
W20260709 13:11:14.515265 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.471576 30931 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.516230 30997 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:14.517113 30793 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.518298 30994 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:14.436429 30812 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:14.518182 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1398) took 1221 ms. Trace:
I20260709 13:11:14.555155 29153 raft_consensus.cc:614] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:14.575709 29078 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:14.534581 30982 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.588680 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1858) took 1369 ms. Trace:
I20260709 13:11:14.589174 29159 rpcz_store.cc:276] 0709 13:11:13.219166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.219288 (+   122us) service_pool.cc:224] Handling call
0709 13:11:13.219562 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:13.380701 (+161139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.587922 (+1207221us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54003edfd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:14.588453 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368899968}
I20260709 13:11:14.591199 30982 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.471592 30504 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.592465 30812 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.594952 30959 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.567492 29085 rpcz_store.cc:276] 0709 13:11:13.296618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.297856 (+  1238us) service_pool.cc:224] Handling call
0709 13:11:14.516696 (+1218840us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:14.517105 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:14.517539 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:14.517578 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:14.517656 (+    78us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:14.517697 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.517772 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.517948 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5598464}
I20260709 13:11:14.598160 30945 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.600381 30793 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.602979 30577 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.538930 30913 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:14.533721 30972 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.501503 30924 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.518805 30605 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:14.607740 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1397) took 1365 ms. Trace:
I20260709 13:11:14.405488 30778 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.552234 30984 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.529371 30923 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.536505 30992 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.500092 30780 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.615631 29078 rpcz_store.cc:276] 0709 13:11:13.242521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.242764 (+   243us) service_pool.cc:224] Handling call
0709 13:11:14.515415 (+1272651us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:14.515792 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:14.581995 (+ 66203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.607684 (+ 25689us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2629888}
I20260709 13:11:14.614301 30793 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.612768 30812 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.528915 30550 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.603801 30577 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.639859 30945 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:14.647199 30605 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.616997 30797 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.525300 30921 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.653218 30981 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.650897 30982 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.649266 30706 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:14.670594 30945 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.671945 30982 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.673697 30945 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:14.673897 30813 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.821s	user 0.014s	sys 0.013s
I20260709 13:11:14.674316 30813 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:14.674746 30982 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:14.670658 30992 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.651086 30997 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:14.677131 30829 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.851s	user 0.002s	sys 0.032s
I20260709 13:11:14.677946 30797 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.662828 30993 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.542028 29079 rpcz_store.cc:276] 0709 13:11:13.225453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.225776 (+   323us) service_pool.cc:224] Handling call
0709 13:11:13.226128 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:13.249890 (+ 23762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:13.249930 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:13.270364 (+ 20434us) log.cc:844] Serialized 52 byte log entry
0709 13:11:13.625418 (+355054us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:13.625461 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.625535 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:14.513385 (+887850us) raft_consensus.cc:1685] finished
0709 13:11:14.513436 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.513674 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25825,"raft.queue_time_us":36839,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9617,"thread_start_us":300414,"threads_started":1,"wal-append.queue_time_us":495078}
I20260709 13:11:14.592533 30994 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.696231 30975 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.662272 30812 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:14.699156 30921 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.699937 30674 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.700210 29001 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:14.700649 30674 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.706184 30993 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.707062 30998 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:14.708606 30941 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.533s	user 0.002s	sys 0.010s
I20260709 13:11:14.668373 30924 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.708988 30993 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.686218 30984 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.712607 30924 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.664559 30778 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:14.713732 30984 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:14.686061 30972 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.686857 30550 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.669853 30839 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.683879 30797 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:14.688133 30923 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.698570 30812 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.679983 30706 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:14.723685 30972 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.725426 29007 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:14.700650 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:14.726647 30577 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.729005 30605 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.729692 30504 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.685388 30659 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:14.731948 30961 log.cc:927] Time spent T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 1.321s	user 0.007s	sys 0.008s
I20260709 13:11:14.728335 30959 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:14.737164 30786 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3.
I20260709 13:11:14.709841 30994 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:14.739868 30964 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 1.284s	user 0.001s	sys 0.000s
W20260709 13:11:14.727885 30335 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.968s	user 0.012s	sys 0.017s
I20260709 13:11:14.681152 30981 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.723512 30948 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:14.759366 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:14.679030 30997 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.762864 29006 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:14.768121 30659 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.748391 30550 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:14.707903 30998 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.784771 29001 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:14.786294 30659 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.786561 30998 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.788219 30812 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.803428 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 162) took 2170 ms. Trace:
I20260709 13:11:14.792400 30550 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.793632 30992 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:14.794631 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1392) took 1570 ms. Trace:
I20260709 13:11:14.789995 30913 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.803911 29156 rpcz_store.cc:276] 0709 13:11:12.632435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.632536 (+   101us) service_pool.cc:224] Handling call
0709 13:11:12.632822 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:12.684009 (+ 51187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:12.684080 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:12.944004 (+259924us) spinlock_profiling.cc:243] Waited 9.24 ms on lock 0x7b18003633a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a
0709 13:11:12.944743 (+   739us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b2c0012c0f8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:12.945167 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:11:13.043685 (+ 98518us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:13.043969 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 13:11:13.231455 (+187486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.688999 (+457544us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b540013c754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:13.689047 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:14.707625 (+1018578us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:14.797059 (+ 89434us) raft_consensus.cc:1685] finished
0709 13:11:14.797101 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.803229 (+  6128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":254355,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":80247,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":15979,"spinlock_wait_cycles":179935488,"thread_start_us":236866,"threads_started":2,"wal-append.queue_time_us":197205}
I20260709 13:11:14.826225 30999 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:14.808144 30965 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 1.048s	user 0.004s	sys 0.015s
I20260709 13:11:14.812192 30706 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.827069 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:14.816372 30550 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.819304 30992 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:14.820089 30889 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.820178 30786 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:14.827965 29078 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:14.833317 30997 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.833577 30786 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.834085 30997 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.835323 30913 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.835489 30808 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.837419 30992 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.840250 30980 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.827085 30999 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:11:14.840337 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1067) took 2655 ms. Trace:
I20260709 13:11:14.838389 29006 raft_consensus.cc:2410] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:11:14.842032 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:14.827991 30706 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:14.841133 30980 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.845667 29156 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:14.835824 30786 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.841614 29155 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:14.809739 29080 rpcz_store.cc:276] 0709 13:11:13.224291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.224416 (+   125us) service_pool.cc:224] Handling call
0709 13:11:13.224807 (+   391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:13.238076 (+ 13269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:13.238108 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:13.267017 (+ 28909us) log.cc:844] Serialized 52 byte log entry
0709 13:11:13.354440 (+ 87423us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:13.808014 (+453574us) log.cc:844] Serialized 14 byte log entry
0709 13:11:13.808289 (+   275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.808352 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:14.737900 (+929548us) spinlock_profiling.cc:243] Waited 51 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:14.794139 (+ 56239us) raft_consensus.cc:1685] finished
0709 13:11:14.794252 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.794447 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":219,"prepare.queue_time_us":323790,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":22800,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":24910,"spinlock_wait_cycles":117310848,"thread_start_us":461228,"threads_started":2,"wal-append.queue_time_us":133136}
I20260709 13:11:14.830333 30504 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:14.832508 30674 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:14.863629 29085 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:14.851593 30335 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.843998 29157 rpcz_store.cc:276] 0709 13:11:12.184972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.407184 (+222212us) service_pool.cc:224] Handling call
0709 13:11:12.407428 (+   244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:12.407611 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:12.407633 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:12.423202 (+ 15569us) log.cc:844] Serialized 26 byte log entry
0709 13:11:12.625847 (+202645us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:12.658756 (+ 32909us) log.cc:844] Serialized 14 byte log entry
0709 13:11:12.666090 (+  7334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.666189 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:12.942314 (+276125us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b180038ed20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f411971558f 00007f4112754571 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd
0709 13:11:13.411256 (+468942us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b180038ed20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1
0709 13:11:14.837724 (+1426468us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b180038ed20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:11:14.839940 (+  2216us) raft_consensus.cc:1685] finished
0709 13:11:14.839958 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.840150 (+   192us) inbound_call.cc:177] Queueing success response
0709 13:11:14.840454 (+   304us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b180038ed20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112754522 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
Metrics: {"mutex_wait_us":19868,"prepare.queue_time_us":35856,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":7347,"raft.queue_time_us":7439,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":10754,"spinlock_wait_cycles":38149632,"thread_start_us":193149,"threads_started":1,"wal-append.queue_time_us":253852}
I20260709 13:11:14.868719 30914 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:14.869567 29007 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:11:14.868466 30335 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.858501 30997 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.870595 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:14.871011 30932 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.823s	user 0.000s	sys 0.015s
I20260709 13:11:14.871992 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:14.872467 30922 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.872897 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1396) took 1619 ms. Trace:
I20260709 13:11:14.873426 29079 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:14.874413 30937 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:14.875130 30950 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.875293 30786 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:14.875696 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:14.883638 29078 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:14.888432 30999 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:14.889163 30918 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 2.008s	user 0.006s	sys 0.005s
I20260709 13:11:14.860482 30889 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.907114 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:14.886487 29081 rpcz_store.cc:276] 0709 13:11:13.231261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.231382 (+   121us) service_pool.cc:224] Handling call
0709 13:11:13.263796 (+ 32414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:13.290614 (+ 26818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:13.290657 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:13.291350 (+   693us) log.cc:844] Serialized 52 byte log entry
0709 13:11:13.390070 (+ 98720us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:13.390108 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.390186 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:14.805262 (+1415076us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b24000fea38. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f41125da9c4 00007f41125d8acb 00007f4112793dc9 00007f4112793a55 00007f411279309d 00007f41153c510e 00007f41153dc88c 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221
0709 13:11:14.830040 (+ 24778us) raft_consensus.cc:1685] finished
0709 13:11:14.830085 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.850543 (+ 20458us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18232,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"spinlock_wait_cycles":44865920,"thread_start_us":88635,"threads_started":2,"wal-append.queue_time_us":351700}
I20260709 13:11:14.899484 30659 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:14.911752 30659 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:14.912403 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.913261 31005 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.913761 31005 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.914587 30659 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.894381 30504 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.919378 29082 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:14.931208 29080 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:14.901777 30992 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.933920 30980 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:14.924283 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 252071us
I20260709 13:11:14.893069 30937 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.907204 30812 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.940339 29157 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:14.946524 29078 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:14.909475 30950 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.905616 30914 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.938155 30678 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.955708 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:14.941761 30786 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.938582 30997 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:14.957466 30981 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:14.959270 30937 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:14.960237 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:14.961228 29076 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:14.961699 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:14.963408 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:14.964346 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:14.964834 30950 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:14.966034 30914 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:14.968191 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.988327 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:14.976306 30997 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.990116 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:14.971222 30936 raft_consensus.cc:3055] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:14.992434 30936 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:14.992869 29717 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:11:14.993269 29717 pending_rounds.cc:85] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Aborting all ops after (but not including) 4
I20260709 13:11:14.993449 30718 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.994040 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:14.994225 30718 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:14.973711 29083 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:14.997315 29122 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:14.934719 30922 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.001231 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.001768 30922 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.001852 29716 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:15.002452 30980 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:15.003266 30980 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:15.004415 31010 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:11:15.004701 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:14.975965 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.993546 30936 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:14.993443 30997 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.007937 30550 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:15.008656 30980 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.008894 31002 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:14.987553 31005 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.009521 31002 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:15.010434 31000 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.999794 29711 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:14.994958 29715 raft_consensus.cc:2410] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:15.009429 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1872) took 1152 ms. Trace:
I20260709 13:11:15.005625 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.007685 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:15.011288 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.010725 31002 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.011070 31000 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:15.017563 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 142225us
I20260709 13:11:15.015245 29154 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:15.007493 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.020509 29084 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:15.022295 30996 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.022771 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.023012 30786 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.022890 30996 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.023905 30504 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.023989 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:14.993515 29717 pending_rounds.cc:107] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:11:15.024971 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.025723 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.025058 29717 raft_consensus.cc:2889] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:15.029389 29083 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:15.030347 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.030963 29083 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:15.031256 31000 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.009935 30998 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:11:15.031857 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.033316 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.005589 31007 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.042250 29048 leader_election.cc:400] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:15.031934 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1866) took 1274 ms. Trace:
I20260709 13:11:15.033829 30839 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:15.043037 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.042601 29048 leader_election.cc:403] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:11:15.044055 29077 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:15.045066 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.045190 30839 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:15.036003 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.017174 30605 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.041555 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.012813 29159 rpcz_store.cc:276] 0709 13:11:13.856868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.590867 (+733999us) service_pool.cc:224] Handling call
0709 13:11:14.591615 (+   748us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:14.636918 (+ 45303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:14.636963 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:14.648924 (+ 11961us) log.cc:844] Serialized 26 byte log entry
0709 13:11:14.920630 (+271706us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:14.920669 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.920802 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:15.009204 (+ 88402us) raft_consensus.cc:1685] finished
0709 13:11:15.009222 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:15.009351 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24361,"raft.queue_time_us":57528,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":2741,"thread_start_us":266842,"threads_started":1,"wal-append.queue_time_us":284914}
I20260709 13:11:15.049906 29713 raft_consensus.cc:1813] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:15.042771 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.052172 30950 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:15.053803 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.034576 30718 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.055768 28975 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:15.056945 29713 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:15.041065 31000 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:14.978826 30812 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.054539 29157 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:15.060333 29079 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
W20260709 13:11:15.061394 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.064733 29716 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.065652 29078 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:15.050209 30783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.728s	user 0.018s	sys 0.015s
I20260709 13:11:15.052191 30839 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:14.988451 30678 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.042613 31007 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.045972 30335 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.084616 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.085233 29083 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
I20260709 13:11:15.100395 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.050483 29076 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.102157 29079 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:11:15.043090 29150 rpcz_store.cc:276] 0709 13:11:13.753568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.753681 (+   113us) service_pool.cc:224] Handling call
0709 13:11:13.753977 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:13.754424 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:13.798653 (+ 44229us) log.cc:844] Serialized 14 byte log entry
0709 13:11:14.538120 (+739467us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b180038ed80. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:11:14.932400 (+394280us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:14.998523 (+ 66123us) log.cc:844] Serialized 26 byte log entry
0709 13:11:14.998679 (+   156us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:14.998717 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.998798 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:15.011734 (+ 12936us) raft_consensus.cc:1685] finished
0709 13:11:15.017718 (+  5984us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:15.028015 (+ 10297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14338,"prepare.queue_time_us":45692,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2489,"raft.queue_time_us":29906,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":13166,"spinlock_wait_cycles":10955008,"thread_start_us":343881,"threads_started":1,"wal-append.queue_time_us":1123180}
W20260709 13:11:15.079742 30783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.905s	user 0.018s	sys 0.015s
I20260709 13:11:15.085851 30659 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.109913 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.110129 30659 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.110687 29711 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.111845 30996 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.112195 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.114707 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.106580 29715 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.116098 30659 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.113335 30391 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:15.125301 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.116786 31007 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.117080 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1855) took 2080 ms. Trace:
I20260709 13:11:15.118211 29710 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:15.113984 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.104224 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.125976 29078 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:15.138013 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.107291 30972 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.129379 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:15.139895 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.141288 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.142728 30889 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.128001 29002 rpcz_store.cc:276] 0709 13:11:13.036981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.038036 (+  1055us) service_pool.cc:224] Handling call
0709 13:11:13.038299 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:13.038486 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:13.122967 (+ 84481us) log.cc:844] Serialized 14 byte log entry
0709 13:11:14.426611 (+1303644us) spinlock_profiling.cc:243] Waited 8.67 ms on lock 0x7b18001c82e0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a
0709 13:11:14.947857 (+521246us) log.cc:844] Serialized 14 byte log entry
0709 13:11:15.028837 (+ 80980us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:15.050691 (+ 21854us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:15.056615 (+  5924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.081599 (+ 24984us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:15.116892 (+ 35293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6463,"prepare.queue_time_us":610034,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":77,"raft.queue_time_us":59798,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":32187,"spinlock_wait_cycles":19936640,"thread_start_us":784756,"threads_started":5,"wal-append.queue_time_us":351159,"wal-append.run_cpu_time_us":2299,"wal-append.run_wall_time_us":1073842}
I20260709 13:11:15.149075 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:15.149643 31018 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:11:15.126461 30391 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.149905 29154 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.150195 30391 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:15.150971 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.151530 29154 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:15.151664 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.152369 29152 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:15.152498 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.153299 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.153617 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:15.154506 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.154706 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.155838 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.157004 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.157800 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:15.157985 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.159077 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.159308 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.124625 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.161201 30812 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.086771 30922 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:11:15.155771 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.175450 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.167359 30783 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:15.168279 30504 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.178767 30783 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.178893 30504 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.179234 30783 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:15.179352 30504 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.143956 30924 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:15.180516 30924 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:15.180903 30924 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:15.100683 30605 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.168856 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.169531 29150 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 1.
I20260709 13:11:15.140489 30998 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:11:15.169912 30659 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:15.168303 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.171095 30936 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:15.172677 30678 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.171782 30921 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.185863 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.191365 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.203867 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.206610 29150 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.179256 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.211784 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.213833 29085 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:15.189965 31000 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:15.213363 29152 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
W20260709 13:11:15.214855 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 107914us
I20260709 13:11:15.214120 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.218262 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.219201 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:15.222720 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:15.211082 30659 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.226218 30659 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.226622 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:15.226909 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.227625 29085 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:15.228519 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.229796 29085 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:15.230185 31025 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.230708 31025 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.232012 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.232610 29716 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:15.232805 31025 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.217399 29076 raft_consensus.cc:2376] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5. Re-sending same reply.
I20260709 13:11:15.233578 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.212713 30921 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.234655 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.235702 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.236107 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.248450 30996 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:15.248558 31000 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.235982 30761 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:15.251428 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:15.240651 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.239598 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.236027 29048 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:15.242660 30605 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.243178 30812 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:15.246168 30998 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:15.248027 29154 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:15.239682 29156 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:15.254222 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.255189 29716 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:15.209396 30922 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.255535 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:15.260165 30922 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:15.262055 30922 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.268473 29079 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.269546 31025 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.270021 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:15.273427 30659 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:15.270138 31025 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.278570 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.279299 29716 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:15.279742 31025 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.280109 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.280427 31025 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:15.280732 29716 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:15.280968 31025 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.282223 30829 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:15.282698 30829 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:15.283205 31025 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.223131 30678 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.275622 30659 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.284638 30678 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.289983 30678 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.288232 30659 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:15.292014 29710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 302) took 2241 ms. Trace:
I20260709 13:11:15.292439 29710 rpcz_store.cc:276] 0709 13:11:13.050166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.051799 (+  1633us) service_pool.cc:224] Handling call
0709 13:11:13.695864 (+644065us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:13.696078 (+   214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:15.135540 (+1439462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.291726 (+156186us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400577bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:15.291938 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3269681152}
I20260709 13:11:15.211256 30972 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.308418 30659 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.279860 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52580 (request call id 6) took 1578 ms. Trace:
I20260709 13:11:15.297250 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.297483 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.298677 29156 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.300215 30931 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:15.300482 29682 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.311463 29153 rpcz_store.cc:276] 0709 13:11:13.701718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.702740 (+  1022us) service_pool.cc:224] Handling call
0709 13:11:15.279564 (+1576824us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b540041a0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583b853 00007f411a80960a 00007f4113d8f7d4 00007f4113d8f762 00007f4113d8f691 00007f4113d8f60c 00007f4113d8e8c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:15.279705 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1857948544}
I20260709 13:11:15.303830 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.275470 30996 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:15.306334 30585 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:15.307906 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.319370 29717 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:15.283682 30829 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.283780 31025 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:11:15.322047 29709 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:15.335227 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.338488 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.339848 29682 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.319288 30605 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:15.342970 29009 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:15.318830 30931 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:15.347965 28975 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:15.400082 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52181.6us
I20260709 13:11:15.342455 29682 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:15.390239 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 139756us
W20260709 13:11:15.408002 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:15.359774 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 123855us
I20260709 13:11:15.362530 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.373436 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.412314 29153 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:15.391963 30923 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.343842 30889 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.333302 30829 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:15.417740 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 47120.2us
I20260709 13:11:15.343242 30996 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.354007 30701 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:15.416745 30921 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.349625 30985 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:15.352656 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.343986 30605 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:15.422288 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 79915.2us
I20260709 13:11:15.349495 30778 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.353993 30659 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.371443 30972 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.456584 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.353844 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:15.493346 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.494998 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.461635 30839 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.496008 30335 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.496542 30335 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.496986 30335 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.497823 30839 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.465989 30701 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:15.499193 30701 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.490432 29003 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:15.500305 30701 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.501606 30990 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.502080 30990 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.502480 30990 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.455758 30783 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.504424 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:15.505154 29157 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:15.505779 29714 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:15.506009 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.506729 29157 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:15.453505 30675 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:15.353365 30637 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.508688 30637 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.409698 30939 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:15.463578 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61792us
I20260709 13:11:15.464390 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:15.459561 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:15.420814 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 147073us
W20260709 13:11:15.476897 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.481807 30605 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:15.421613 30889 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.472422 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.454118 31004 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:15.423768 30479 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:15.481878 30716 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:15.416393 30922 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.490391 30391 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.513468 30794 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.545567 30922 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.539021 30972 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:15.546206 30972 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.546078 30794 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.546641 30972 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.515426 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.514776 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.548426 29006 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:15.549327 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.550017 29006 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:15.550688 30761 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.550870 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.516119 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.551224 30761 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.518540 30921 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:15.552778 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:15.521162 30956 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 2.066s	user 0.006s	sys 0.000s
I20260709 13:11:15.459923 30947 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.487310 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.528811 28975 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.486263 30778 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:15.540815 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:15.542908 31019 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:11:15.546525 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.549475 30950 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.551615 29001 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:15.516803 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1091) took 2515 ms. Trace:
I20260709 13:11:15.517810 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.520974 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:15.535145 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 62857us
W20260709 13:11:15.568274 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.466706 30504 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.545640 30391 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.556977 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.537927 30889 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.538089 30479 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.551256 29711 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:15.490033 30996 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.471678 30923 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.558699 29007 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:15.591637 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.549327 30794 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.472615 31002 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:11:15.567384 30839 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:15.572927 30778 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:15.589260 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 60504.2us
I20260709 13:11:15.482856 30931 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.602450 30810 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.603189 30931 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.603014 30810 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.582629 29006 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:15.575088 30659 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:15.602131 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.602244 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:15.570564 30637 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.600673 29005 pending_rounds.cc:85] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:11:15.609726 29005 pending_rounds.cc:107] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:11:15.588469 30479 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:15.610013 29005 raft_consensus.cc:2889] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:15.606943 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.610674 29712 raft_consensus.cc:2368] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5.
I20260709 13:11:15.596685 30761 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.599860 30839 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.621858 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.600719 31000 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.622430 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:15.623718 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.631485 29007 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:15.632509 29008 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:15.636255 30996 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.638599 30923 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.639624 30780 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.577701 29155 rpcz_store.cc:276] 0709 13:11:13.001661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.013556 (+ 11895us) service_pool.cc:224] Handling call
0709 13:11:14.840914 (+1827358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:14.862786 (+ 21872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.516522 (+653736us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54003ec6d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:15.516732 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4167342976}
I20260709 13:11:15.593012 30922 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:11:15.642351 29082 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:15.632896 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:15.642144 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:15.650158 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 49681.5us
I20260709 13:11:15.600114 30950 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.650547 29078 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:15.654021 29680 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.654150 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.651757 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.649976 30985 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.656435 30985 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.625867 30839 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:15.659010 28975 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:15.631031 30391 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:15.662781 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:15.634795 31033 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:11:15.666234 30958 raft_consensus.cc:476] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Not starting pre-election -- already a leader
W20260709 13:11:15.666754 31026 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:11:15.667312 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:15.670125 31032 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:11:15.670719 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.671258 29151 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:15.673067 30950 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.665462 28975 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:15.584904 30947 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.684628 29158 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:15.660363 30996 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:15.720835 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.690857 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.681200 30659 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:15.681346 31007 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:15.723296 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 57905.4us
I20260709 13:11:15.695518 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.695765 30972 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.696722 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.697394 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.725620 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.700389 29712 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:11:15.701634 30924 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.001s
I20260709 13:11:15.702543 30923 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:15.632638 31000 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.707476 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.708278 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.708395 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.709064 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.731807 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:15.735869 29006 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:15.709223 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.710050 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.710759 29003 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:15.710676 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:15.737615 29079 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:15.738721 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.739277 29079 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:15.711174 30391 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.713534 30957 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.717006 29714 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:15.676890 30937 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.692395 30675 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:15.727298 28975 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:15.746767 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.728349 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.686760 29077 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:15.734242 29082 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:11:15.710788 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.748103 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.723841 30659 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.748831 29010 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:15.745451 29001 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:15.744259 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.726804 30504 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.727977 30637 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.747466 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.752542 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.754251 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.754377 29083 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:15.730856 30923 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.754741 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.756433 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.747565 29008 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:15.755324 29084 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:15.765208 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.765879 29079 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:15.738574 31001 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.751595 29710 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:15.811018 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.772686 29001 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:15.734544 30924 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.753654 30939 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:15.803236 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:15.803371 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 56006.6us
I20260709 13:11:15.751879 30996 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:15.752142 30659 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:15.811088 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.779227 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.837720 29080 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:15.744112 30391 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.732815 30839 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.764175 30923 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:15.805083 30675 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:15.831748 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.849699 29001 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:15.777532 30504 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:15.834901 30682 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:11:15.835199 31033 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:15.815438 30637 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.832489 29085 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:15.767865 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.838244 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.845531 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.863821 29083 raft_consensus.cc:2376] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3. Re-sending same reply.
I20260709 13:11:15.771497 30985 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.766306 30810 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.830500 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.832159 28975 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.778923 31001 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.815577 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.810830 30585 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:11:15.850997 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:15.875723 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 221743us
I20260709 13:11:15.852900 29081 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
W20260709 13:11:15.854089 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 260543us
I20260709 13:11:15.876495 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:15.877560 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 45518.1us
I20260709 13:11:15.878237 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:15.878651 28975 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:15.767441 30937 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.880894 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.747428 30957 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.884662 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.839169 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.856701 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.855263 30504 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.871196 29006 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:15.877933 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.875945 29003 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:15.885535 29010 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:15.890164 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.886215 29084 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:15.891286 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.891824 29084 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:15.873703 31001 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.896997 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.898666 29078 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:15.877353 30985 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:15.899717 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.889118 30675 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:15.906177 29082 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:15.891467 30996 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.892539 30504 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:15.888413 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.880543 29076 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:15.902402 29078 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:15.911691 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:15.911813 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.920800 29001 raft_consensus.cc:2410] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:11:15.901733 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.900892 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.910708 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.915697 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.924141 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:15.911487 30701 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:11:15.936976 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 183794us
I20260709 13:11:15.937326 29680 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:15.898617 31042 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:15.933529 29085 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:15.913545 30996 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:15.941206 31039 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:11:15.953354 30391 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.953197 30778 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.940256 30718 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:15.954690 30718 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:15.955605 29713 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:15.955562 30718 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.900980 30985 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:15.956947 30985 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:15.957355 31040 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:15.958554 31007 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:15.946619 30839 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.925818 30972 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:15.959038 31007 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:15.948271 30947 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.942206 31042 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.962284 31042 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.964396 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.964640 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.965441 29076 raft_consensus.cc:2410] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:11:15.965514 29084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "243d4ae4a4ae48b2979e32874472d256"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:15.966300 29084 raft_consensus.cc:606] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Received request to transfer leadership to TS 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:15.967717 29084 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:15.956126 30335 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:15.957007 30778 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:15.944629 31037 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:11:15.945655 30637 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:15.968348 29080 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:15.968573 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1855) took 2956 ms. Trace:
I20260709 13:11:15.918314 30675 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:15.969350 30810 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.939507 30701 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:15.925316 30716 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:15.970391 31038 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:11:15.959038 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.941609 30950 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:11:15.975550 28976 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.976511 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:15.978117 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.979871 29151 rpcz_store.cc:276] 0709 13:11:13.011751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.018414 (+  6663us) service_pool.cc:224] Handling call
0709 13:11:13.018680 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:15.688286 (+2669606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.968298 (+280012us) spinlock_profiling.cc:243] Waited 2.65 s on lock 0x7b54003edad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:15.968519 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6098784000}
I20260709 13:11:15.988309 29082 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:15.988417 29084 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:15.984890 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:15.989797 29009 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.975989 30996 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:15.988826 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:15.991071 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:15.939162 30797 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:15.987458 29002 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:15.943784 31000 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:16.007261 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:15.992084 30335 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:15.990485 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:15.994669 30780 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:15.994607 30701 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.995378 30939 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:16.046445 30939 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:16.046828 30939 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:15.996017 30810 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.048501 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.049787 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.050069 30810 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.050318 29003 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:16.051249 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.998401 29714 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:15.999142 30675 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:15.999922 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.000581 30937 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:16.001461 29009 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:15.983395 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:16.030676 29004 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:15.915370 30990 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.033756 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:16.006485 29158 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:15.993959 30924 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:16.035357 29717 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:15.991350 30839 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:16.039969 31041 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.039891 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.028723 30391 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:16.002557 30936 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:16.044572 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.044723 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:15.997421 30811 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:16.052316 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:15.997758 30950 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:16.051932 30889 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.053197 30605 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.055382 30718 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.060436 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:16.063104 29082 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
W20260709 13:11:16.065538 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65686us
I20260709 13:11:16.086294 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:16.069309 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 172) took 1131 ms. Trace:
I20260709 13:11:16.069856 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.072721 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:16.055302 31007 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.040690 31050 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:16.074877 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.039264 30812 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
W20260709 13:11:16.075577 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.078222 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.078372 29085 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:16.092895 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.092916 29001 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:16.094327 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.076939 29680 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:16.082186 29077 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:16.066741 30780 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:16.099870 29009 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:16.100279 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:16.071913 31041 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.090088 29007 raft_consensus.cc:2410] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:11:16.047667 30939 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:16.095376 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.091892 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:16.120213 29010 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:16.087124 29158 rpcz_store.cc:276] 0709 13:11:14.934582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.955293 (+ 20711us) service_pool.cc:224] Handling call
0709 13:11:15.667083 (+711790us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ceee 0000000000375939 00007f411a7901e0 00007f411a8590c4 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:16.065594 (+398511us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39272704}
I20260709 13:11:16.101224 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.102104 29082 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:11:16.122740 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.099778 30637 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:16.103421 30936 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:16.103369 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.130595 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.085952 30950 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:16.106027 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:16.106806 30783 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.100056 30921 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:16.107445 30931 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:16.107963 30780 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.108570 31051 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:16.110697 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.110903 30990 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:16.111125 30479 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.111125 29716 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
W20260709 13:11:16.113235 31046 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:11:16.166846 30479 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:16.167387 30996 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.114519 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.170320 29001 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:16.083666 30957 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:16.105230 31007 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:16.133153 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:16.089684 30391 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.171949 30921 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:16.172210 30391 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:16.172646 30391 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.172981 30921 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.097674 30701 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:16.154434 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.177546 29082 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.154500 29680 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:16.162782 30937 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:16.164945 29007 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:16.153925 31051 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:16.114061 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.134277 30637 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:16.118075 31041 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.150288 30811 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:16.180730 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 47669.9us
I20260709 13:11:16.197005 29006 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:16.160789 30839 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:16.195693 29081 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:16.101125 31050 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.175510 30939 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.195101 31051 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.197531 30637 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:16.193322 31049 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 13:11:16.208740 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 114492us
I20260709 13:11:16.210534 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:16.213430 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.213799 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.200158 30972 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:16.216481 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:16.218202 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.198745 30811 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:16.224802 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.226632 28976 leader_election.cc:400] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5.
I20260709 13:11:16.226940 28976 leader_election.cc:403] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:11:16.205207 30924 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:16.243026 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.246703 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:16.257979 29082 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:16.216221 30957 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:16.260895 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.226717 30811 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.266131 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:16.264859 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:16.260319 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 184514us
I20260709 13:11:16.242019 30812 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:16.250560 30990 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:16.253532 30839 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:16.281039 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.250370 30972 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:16.265512 30479 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:16.283967 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:16.210527 30950 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.294428 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:16.296273 29714 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:16.249418 31033 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5.
I20260709 13:11:16.267639 31007 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:16.301066 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:16.283259 29122 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:16.346913 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:16.283809 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:11:16.349607 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 48608.4us
I20260709 13:11:16.262219 30923 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:16.350337 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 67147.5us
I20260709 13:11:16.291152 31060 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:16.387962 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 83596us
I20260709 13:11:16.387830 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:16.347590 29083 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:16.387805 30990 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:16.295775 30479 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:16.395402 29082 raft_consensus.cc:2410] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:16.268926 30924 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.390447 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:16.396564 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:16.392697 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 114603us
W20260709 13:11:16.397923 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 147104us
I20260709 13:11:16.395308 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:16.393874 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:16.400235 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.349602 30889 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.290373 31001 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.295328 30812 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 13:11:16.391225 30957 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.405383 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:16.407187 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.429348 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.422637 29715 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:16.419641 30391 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:16.412326 30659 consensus_meta.cc:287] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.550s	user 0.002s	sys 0.007s
I20260709 13:11:16.412765 30585 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:16.413273 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:16.396251 31060 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:16.414794 31055 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 13:11:16.415642 31057 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:11:16.416051 31061 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:16.397472 30479 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.396876 30950 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:16.433943 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:16.388263 31062 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.436996 29004 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:16.406739 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:16.397504 30990 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.346131 30985 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.399405 31007 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:16.541407 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 134698us
I20260709 13:11:16.542375 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:16.419622 30947 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.490065 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.544849 29083 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.491196 30391 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.407333 30996 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.404677 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.433259 30957 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:16.507508 29008 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:16.493695 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.533211 30794 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:16.535996 29709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 515) took 1016 ms. Trace:
I20260709 13:11:16.536748 31050 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:16.509721 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.487313 31027 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.549675 29002 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:16.488059 31060 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.540910 30990 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:16.560119 29079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.026s	sys 0.015s
I20260709 13:11:16.486227 30659 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.479681 30839 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:16.561022 29079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.026s	sys 0.015s
I20260709 13:11:16.544071 31007 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.540586 31062 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.551841 30957 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.567993 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.569705 29005 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:16.569963 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:16.570844 29002 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:16.551389 30794 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:16.573099 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 159900us
I20260709 13:11:16.489009 30972 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.575641 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.577044 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.567870 31007 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.567898 31062 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.578043 30794 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:16.548902 30637 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.579182 29003 raft_consensus.cc:2393] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:16.628985 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 228797us
W20260709 13:11:16.628198 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 52638.8us
I20260709 13:11:16.634699 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:16.597886 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 201764us
W20260709 13:11:16.636713 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.586952 29006 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:16.589262 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:16.630606 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:16.632025 29683 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:16.589424 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:16.585951 31063 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
W20260709 13:11:16.588071 31059 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:11:16.638775 29084 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
W20260709 13:11:16.641512 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 99215.8us
I20260709 13:11:16.656920 29083 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:16.583665 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.597469 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:16.661008 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.596549 30923 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:16.638715 30659 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.654839 30675 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:16.640604 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:16.666318 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.594290 30924 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.668678 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:16.638188 30761 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:16.670243 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.671661 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.677800 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:16.663028 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:16.678552 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.553058 29709 rpcz_store.cc:276] 0709 13:11:15.519827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:15.537708 (+ 17881us) service_pool.cc:224] Handling call
0709 13:11:16.136343 (+598635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:16.165178 (+ 28835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:16.165216 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:16.165287 (+    71us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:16.165332 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:16.165392 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:16.535795 (+370403us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54007409d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:16.535924 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218497792}
I20260709 13:11:16.687316 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:16.688371 29082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31c5d53651b5426780911903843f0bc3"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:16.688668 31079 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.688926 29082 raft_consensus.cc:606] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:16.689138 31079 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.689335 29082 raft_consensus.cc:614] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:16.689540 31079 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.676600 31064 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.668193 31068 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:16.672508 29076 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.685993 29006 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:16.681910 30682 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:16.598048 31067 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:11:16.637713 30889 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:11:16.686087 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:16.686075 29084 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:16.696107 30780 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:11:16.696806 30504 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.771s	user 0.012s	sys 0.002s
W20260709 13:11:16.676129 31076 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:11:16.683045 31042 consensus_meta.cc:287] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.720s	user 0.014s	sys 0.000s
I20260709 13:11:16.578622 30839 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.674542 31050 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.707574 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:16.697779 31072 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:16.686548 31073 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:11:16.730547 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:16.685225 30996 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:11:16.731038 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60854.4us
I20260709 13:11:16.697597 31064 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:16.737430 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1013) took 1020 ms. Trace:
I20260709 13:11:16.739835 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.740852 30391 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:16.741979 29080 rpcz_store.cc:276] 0709 13:11:15.716509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:15.980011 (+263502us) service_pool.cc:224] Handling call
0709 13:11:15.982465 (+  2454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:16.032232 (+ 49767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:16.032289 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:16.039519 (+  7230us) log.cc:844] Serialized 26 byte log entry
0709 13:11:16.211008 (+171489us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:16.211061 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:16.211148 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:16.737164 (+526016us) raft_consensus.cc:1685] finished
0709 13:11:16.737192 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:16.737348 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46936,"raft.queue_time_us":56820,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":125101,"threads_started":1,"wal-append.queue_time_us":215582}
I20260709 13:11:16.709257 30659 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:16.715621 29710 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:16.728904 31050 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.710932 30780 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:16.747973 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:16.754997 29158 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:16.736690 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:16.752324 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:16.753016 29150 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:16.734687 30675 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:16.756068 29156 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:16.732208 31072 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.738808 30682 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.744484 31000 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:16.729861 31042 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.770787 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:16.768419 29080 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:16.775995 29080 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:16.744221 31064 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.771371 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:16.775449 29154 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:16.720098 31041 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.755223 30985 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.757457 31050 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.734623 30924 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:11:16.773619 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:16.776912 29080 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.790855 29080 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.730281 30504 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:16.782048 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 150097us
I20260709 13:11:16.784642 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:16.773918 30659 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:16.788749 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:16.770898 31072 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.767923 30637 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:16.783011 30984 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:16.800007 29713 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:11:16.784368 31085 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:16.726938 31068 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.801564 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:16.804253 29717 raft_consensus.cc:2393] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:16.743935 31070 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:11:16.798704 29158 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
W20260709 13:11:16.795138 31051 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.003s
I20260709 13:11:16.770471 30682 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:16.804703 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 64975.8us
I20260709 13:11:16.802465 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:16.820662 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:16.820914 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:16.819660 30391 raft_consensus.cc:697] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:16.823288 29001 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:16.825907 29001 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:16.827564 29157 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:16.803725 30659 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:16.802177 31088 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:16.807549 30996 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:16.828971 30391 consensus_queue.cc:237] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.805260 30888 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:16.828791 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 103488us
I20260709 13:11:16.793382 31054 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:11:16.784214 30811 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.820143 30682 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.820187 30637 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:16.827338 31082 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:16.843636 30659 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.854425 30888 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.841502 31084 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:16.871320 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:16.873780 29157 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:16.882328 29080 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:11:16.882690 29080 pending_rounds.cc:85] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 3
I20260709 13:11:16.882939 29080 pending_rounds.cc:107] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:11:16.883168 29080 raft_consensus.cc:2889] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:16.827566 31060 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.813557 30780 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.828146 31051 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.865515 31088 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:16.880340 30701 consensus_meta.cc:287] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.002s
I20260709 13:11:16.821085 30984 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:16.873596 30888 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:16.892432 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:16.866945 31090 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:16.821214 31085 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.884440 31082 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:16.879685 31002 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:16.893918 31083 log.cc:927] Time spent T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:11:16.906518 29159 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:16.873792 30637 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.890686 30839 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:16.918483 31066 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:11:16.920574 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 147008us
I20260709 13:11:16.895138 31041 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:16.927488 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:16.949620 31050 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.914336 30780 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.965394 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1431) took 1047 ms. Trace:
I20260709 13:11:16.914711 31082 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.906131 31099 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:16.966094 30936 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:16.933069 29122 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.768s (SENT) [suppressed 26 similar messages]
I20260709 13:11:16.965833 29078 rpcz_store.cc:276] 0709 13:11:15.918759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.662092 (+743333us) service_pool.cc:224] Handling call
0709 13:11:16.662475 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:16.693994 (+ 31519us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:16.778858 (+ 84864us) log.cc:844] Serialized 14 byte log entry
0709 13:11:16.965013 (+186155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:16.965071 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:16.965131 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:16.965196 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:16.965333 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70710,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":28,"raft.queue_time_us":33067,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":10061,"thread_start_us":241382,"threads_started":3,"wal-append.queue_time_us":101934}
I20260709 13:11:16.928784 30761 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:16.973498 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 304874us
I20260709 13:11:16.929746 31066 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:16.973935 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:16.971472 29125 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:16.978148 29157 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:16.976445 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:16.962175 31042 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:16.891374 31084 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.917222 31002 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:16.973212 31082 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:16.991900 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:17.011415 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.975800 30504 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:17.069957 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 198) took 1087 ms. Trace:
W20260709 13:11:17.067983 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 69065.9us
I20260709 13:11:16.916996 30996 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.079885 29153 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:17.075753 29009 rpcz_store.cc:276] 0709 13:11:15.982617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.226137 (+243520us) service_pool.cc:224] Handling call
0709 13:11:16.260267 (+ 34130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:16.399442 (+139175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:16.399477 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:16.410694 (+ 11217us) log.cc:844] Serialized 26 byte log entry
0709 13:11:16.574464 (+163770us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:16.574504 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:16.574572 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:17.069664 (+495092us) raft_consensus.cc:1685] finished
0709 13:11:17.069692 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:17.069855 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12175,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":204412,"threads_started":2,"wal-append.queue_time_us":216389}
I20260709 13:11:16.973505 31106 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:17.094915 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:17.095541 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.973827 30936 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.099105 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.070555 31067 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:17.111850 29157 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:17.114290 30637 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:17.114892 30637 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:17.115236 30637 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:17.115577 30637 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:17.107196 31007 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.007s
I20260709 13:11:17.125816 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.134482 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:17.135237 29157 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:17.139822 31007 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.140993 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.143076 31050 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:17.144398 31104 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:11:17.145682 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1023) took 1165 ms. Trace:
I20260709 13:11:17.146034 29083 rpcz_store.cc:276] 0709 13:11:15.980472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.667141 (+686669us) service_pool.cc:224] Handling call
0709 13:11:16.667756 (+   615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:16.683342 (+ 15586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:16.683366 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:16.684470 (+  1104us) log.cc:844] Serialized 556 byte log entry
0709 13:11:16.916526 (+232056us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:16.916572 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:16.920348 (+  3776us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:17.145488 (+225140us) raft_consensus.cc:1685] finished
0709 13:11:17.145505 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:17.145618 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":223233,"threads_started":1,"wal-append.queue_time_us":271477}
W20260709 13:11:17.144495 31100 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
W20260709 13:11:17.149852 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 216772us
I20260709 13:11:17.064615 30675 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:17.143831 28975 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:17.152941 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:17.122040 30811 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:17.161046 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 216) took 1390 ms. Trace:
I20260709 13:11:17.134320 31060 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:17.163131 29085 rpcz_store.cc:276] 0709 13:11:15.770735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.099690 (+328955us) service_pool.cc:224] Handling call
0709 13:11:16.103011 (+  3321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:16.117856 (+ 14845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:16.117888 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:16.118270 (+   382us) log.cc:844] Serialized 26 byte log entry
0709 13:11:16.251748 (+133478us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:16.251795 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:17.160589 (+908794us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5400146754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:17.160650 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:17.160710 (+    60us) raft_consensus.cc:1685] finished
0709 13:11:17.160730 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:17.160970 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33605376,"thread_start_us":85588,"threads_started":1,"wal-append.queue_time_us":327215}
I20260709 13:11:17.159715 30839 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
W20260709 13:11:17.171658 30479 consensus_meta.cc:287] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.627s	user 0.000s	sys 0.008s
I20260709 13:11:17.172113 30479 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.157418 31099 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:17.175452 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.160001 31007 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:17.173974 30794 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.003s
I20260709 13:11:17.106595 31106 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.159970 31108 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:17.168371 31051 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:17.182660 29009 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:17.175566 31099 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:17.187625 30939 consensus_meta.cc:287] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.977s	user 0.003s	sys 0.008s
I20260709 13:11:17.196201 30888 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.109555 31088 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.201759 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.207636 31072 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:17.211203 31101 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:11:17.211288 31072 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:17.192866 31068 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:17.139523 31066 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.195236 31099 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:17.190208 30794 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.203122 30984 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:17.222038 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.223791 29157 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:17.207656 31084 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:17.230636 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:17.231313 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 190553us
I20260709 13:11:17.221437 31066 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:17.238036 30957 consensus_meta.cc:287] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.001s
I20260709 13:11:17.238832 31027 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:17.242292 30990 consensus_meta.cc:287] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.666s	user 0.001s	sys 0.007s
I20260709 13:11:17.243587 30936 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:17.243727 31106 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:17.251394 31079 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.561s	user 0.003s	sys 0.006s
W20260709 13:11:17.253553 31110 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:11:17.254045 30950 consensus_meta.cc:287] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.000s
I20260709 13:11:17.259775 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.256345 30682 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.247605 31064 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:17.263445 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.289945 31106 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.290349 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:17.290442 31106 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.291584 30761 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:17.291541 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:17.293066 31106 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:17.293468 30718 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.195s	user 0.010s	sys 0.006s
W20260709 13:11:17.257015 30947 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.709s	user 0.005s	sys 0.005s
I20260709 13:11:17.270435 29154 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
W20260709 13:11:17.275075 31062 consensus_meta.cc:287] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.645s	user 0.006s	sys 0.001s
I20260709 13:11:17.275831 29081 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:17.261533 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.260836 31027 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:17.269001 31079 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.280854 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:17.287652 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:17.212175 30675 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.297577 29009 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:17.266505 30990 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.300335 30761 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:17.313208 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.304704 30778 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:17.332731 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 353948us
I20260709 13:11:17.333127 29682 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:17.281550 30479 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:17.336805 29152 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:17.337074 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:17.337605 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:17.282857 30921 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:17.339391 29157 raft_consensus.cc:2368] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:17.342417 29151 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:17.336210 30504 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:17.336400 30984 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:11:17.343775 30585 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:17.344178 30984 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:17.344316 30585 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:17.344697 30984 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:17.345538 30984 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.346834 30984 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.315048 30761 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:17.321058 30947 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:17.265169 30957 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.352090 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.352665 29154 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:17.342150 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.354983 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:17.350808 29076 consensus_meta.cc:287] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.001s
I20260709 13:11:17.304451 31062 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.341059 30479 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:17.350193 31042 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:17.351336 30761 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:17.366813 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:17.366044 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:17.379691 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.381281 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:17.381470 31072 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:17.382094 30839 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:17.382402 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:17.384593 31108 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:17.383615 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.398249 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.400331 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:17.402706 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.403309 29078 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:17.362797 29076 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:17.366021 31088 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:17.402737 30794 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:17.415949 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.416123 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.416289 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:17.419027 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.420143 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:17.421134 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.424317 30682 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:17.425573 29006 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:17.426019 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:17.426832 29709 raft_consensus.cc:2368] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:17.427621 29155 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:17.428419 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.429553 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.430076 29082 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:17.431718 29080 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:17.433426 29711 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:17.386265 31002 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:17.351018 30921 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.407548 30936 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.362202 30659 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.493337 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:17.496496 29157 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:17.496515 29082 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:17.498849 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.419370 31082 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.496336 31107 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:17.404031 29077 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:17.411688 30780 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:17.499552 29080 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:17.508955 31107 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.512665 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.521252 29080 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:17.522483 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:17.523044 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.524540 31079 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:17.527205 30659 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:17.528383 30637 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.531289 30637 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:17.532747 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.533061 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.533325 29010 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.533744 29159 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:17.532788 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.534921 29159 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:17.533643 29002 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:17.519666 31085 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:17.436873 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1031) took 1018 ms. Trace:
I20260709 13:11:17.536410 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:17.537515 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:17.539260 29080 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:17.539932 29153 raft_consensus.cc:2393] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:17.543047 29078 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:17.544212 29683 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:17.538829 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.549467 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:17.523897 29155 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:17.551962 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.543956 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.554708 29078 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:17.558766 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.559113 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:17.554183 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.561015 29001 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:11:17.565358 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.565778 29078 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:17.566849 29078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:17.560678 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:17.567452 29078 raft_consensus.cc:606] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:17.570184 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.570936 29079 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:17.572223 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:17.504179 31118 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:17.568024 30716 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:17.500221 30936 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.574471 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.553797 30637 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:17.579252 29010 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:17.591830 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:17.554294 31122 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 13:11:17.560369 31116 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:11:17.553849 30718 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:17.554450 30972 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:17.590406 29001 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:17.559286 30701 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:17.539381 30780 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.604964 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.605172 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.605576 29005 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:17.600764 29717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47384 (request call id 526) took 1586 ms. Trace:
W20260709 13:11:17.600715 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 51314.6us
I20260709 13:11:17.566803 31066 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:17.606865 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.609977 29009 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:17.599424 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.545440 29076 rpcz_store.cc:276] 0709 13:11:16.417142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.583411 (+166269us) service_pool.cc:224] Handling call
0709 13:11:17.435180 (+851769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:17.606002 29717 rpcz_store.cc:276] 0709 13:11:16.009188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.072458 (+ 63270us) service_pool.cc:224] Handling call
0709 13:11:17.595878 (+1523420us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b5400546ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:17.596022 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1647198976}
I20260709 13:11:17.614248 29125 leader_election.cc:400] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:17.614677 29125 leader_election.cc:403] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:17.591353 30990 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:17.591665 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:17.592516 30659 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.650195 29085 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:17.647029 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.655959 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:17.651352 31097 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.684s	user 0.004s	sys 0.000s
I20260709 13:11:17.669286 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:17.661825 28976 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:17.665501 29079 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:17.663180 29155 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:17.619117 30778 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:17.645161 30972 raft_consensus.cc:697] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:17.683092 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.698997 29009 raft_consensus.cc:2368] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
W20260709 13:11:17.683844 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.659832 31050 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:17.696573 29717 raft_consensus.cc:2368] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:17.645295 31054 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:17.589531 31118 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.708417 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.610956 31066 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.659727 31042 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:17.714745 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:17.701071 29008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1491) took 1849 ms (client timeout 1768 ms). Trace:
W20260709 13:11:17.701445 29081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 225) took 1570 ms (client timeout 1897 ms). Trace:
I20260709 13:11:17.709290 29001 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:17.710043 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.716744 29079 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:17.716239 29081 rpcz_store.cc:269] 0709 13:11:16.114815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.203665 (+ 88850us) service_pool.cc:224] Handling call
0709 13:11:17.685012 (+1481347us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54000451d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:17.685150 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2269009664}
I20260709 13:11:17.699771 31000 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:17.718801 31000 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:17.701766 30937 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:17.656221 30659 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:17.715927 29008 rpcz_store.cc:269] 0709 13:11:15.840655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:15.868310 (+ 27655us) service_pool.cc:224] Handling call
0709 13:11:17.689930 (+1821620us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b5400185bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:17.690078 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1430081792}
I20260709 13:11:17.721131 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.709023 29077 raft_consensus.cc:2410] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:17.726572 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:17.733520 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61529.7us
I20260709 13:11:17.734208 29125 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:17.719089 31123 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:11:17.737542 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:17.740432 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:17.744643 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.745492 29078 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:17.746119 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:17.747020 29125 leader_election.cc:400] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:17.747340 29125 leader_election.cc:403] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:11:17.749056 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:17.691551 30990 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:11:17.734622 30937 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.737053 30637 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:17.702416 30778 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.762921 30990 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:17.763368 30990 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:17.764492 30990 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.765996 30990 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:17.768435 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:17.741402 30718 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:17.738927 30701 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:17.763011 29154 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:17.711534 31054 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.776336 29002 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:17.751485 31085 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:17.724563 31042 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.779083 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.787034 29078 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:17.779912 31127 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:11:17.748651 30921 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:17.795259 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.774444 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.778141 30972 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.785651 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.790156 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.800630 29009 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:17.748234 30957 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:17.772979 30810 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:17.781236 30701 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:17.775506 30637 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:17.802526 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:17.812857 29154 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:17.812898 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.813840 29079 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:17.814396 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:17.818446 29001 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:17.794833 30678 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.798032 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.792178 30718 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:17.841624 29078 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:17.842538 29078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:17.806967 31085 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:17.843075 29078 raft_consensus.cc:606] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:17.811895 30936 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:17.813380 30701 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:17.809016 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:17.848933 29154 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:17.856751 29079 raft_consensus.cc:2410] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:17.857795 29079 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:17.866571 29009 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:17.834578 30780 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:17.870641 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 293788us
I20260709 13:11:17.854979 29155 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:17.885844 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:17.895725 29154 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:17.930222 29005 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:17.910081 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1133) took 1134 ms. Trace:
I20260709 13:11:17.956633 29150 rpcz_store.cc:276] 0709 13:11:16.775382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.777730 (+  2348us) service_pool.cc:224] Handling call
0709 13:11:16.778145 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:17.091461 (+313316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:17.091499 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:17.105504 (+ 14005us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:17.105563 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:17.909841 (+804278us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54002a44d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:17.909893 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:17.910021 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":692485504}
I20260709 13:11:17.841712 30678 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:17.834806 30810 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:17.811403 31000 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:17.967902 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 82150.2us
I20260709 13:11:17.854281 29157 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:17.980487 29079 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:17.989260 29680 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:17.990468 29680 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:17.991279 29680 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:17.988391 30504 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:18.012380 29155 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:18.016935 30605 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:17.989848 30682 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:18.026239 31066 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.027781 30810 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:18.028498 30761 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.000s
I20260709 13:11:18.028805 31042 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.028932 30761 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.031168 30761 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.033051 29712 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:18.041760 31062 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:18.044615 31134 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
W20260709 13:11:18.056488 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 194112us
I20260709 13:11:17.979931 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1898) took 1026 ms. Trace:
I20260709 13:11:18.001286 30921 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:18.068385 29084 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:18.102705 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 319) took 2121 ms. Trace:
I20260709 13:11:18.007638 31041 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:18.013787 31118 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:18.105540 29713 rpcz_store.cc:276] 0709 13:11:15.981126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:15.981244 (+   118us) service_pool.cc:224] Handling call
0709 13:11:15.989615 (+  8371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:16.813838 (+824223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.102417 (+1288579us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54004ffe54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:18.102633 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1862928128}
I20260709 13:11:18.104872 29159 rpcz_store.cc:276] 0709 13:11:16.962558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.724806 (+762248us) service_pool.cc:224] Handling call
0709 13:11:17.725074 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:17.725272 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:17.725567 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:11:17.979188 (+253621us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:17.979497 (+   309us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:17.979569 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:17.979639 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:17.979869 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90219,"prepare.queue_time_us":865,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":9270,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8363,"thread_start_us":253220,"threads_started":1,"wal-append.queue_time_us":215597}
I20260709 13:11:18.066499 30659 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:18.102599 31133 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.253s	user 0.000s	sys 0.000s
I20260709 13:11:18.021245 30984 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:18.156122 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:18.164842 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:18.174095 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:18.222605 29159 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:18.131282 31042 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:18.137538 31054 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:18.020583 30718 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:18.224365 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 59637.9us
I20260709 13:11:18.166447 31041 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:18.055513 31085 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:18.174930 29085 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.000s
W20260709 13:11:18.206173 29003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1489) took 2399 ms (client timeout 2572 ms). Trace:
I20260709 13:11:18.195226 31094 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:18.189329 31137 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:11:17.960526 30637 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.205741 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:18.217852 31001 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.140152 30778 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:18.202541 31099 consensus_meta.cc:287] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.982s	user 0.007s	sys 0.003s
I20260709 13:11:18.231863 29157 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:18.231485 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:18.210237 31061 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.659s	user 0.022s	sys 0.020s
I20260709 13:11:18.203560 29007 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:18.197077 30984 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:18.236883 31098 raft_consensus.cc:3055] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:18.238873 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:18.238822 30701 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.230829 30761 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:18.254961 30761 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:18.255374 30761 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:18.256944 31113 log.cc:927] Time spent T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.061s	user 0.004s	sys 0.008s
I20260709 13:11:18.249929 31066 consensus_queue.cc:1059] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:11:18.258761 30780 raft_consensus.cc:3055] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:18.259116 30780 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:18.259649 31027 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.947s	user 0.009s	sys 0.001s
W20260709 13:11:18.234476 29003 rpcz_store.cc:269] 0709 13:11:15.803526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.087005 (+283479us) service_pool.cc:224] Handling call
0709 13:11:18.202765 (+2115760us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b540035d9d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:18.202957 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043928320}
I20260709 13:11:18.234436 29085 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:18.249897 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:18.263361 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:18.259308 30984 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.259963 30780 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.266741 30780 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:18.266981 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1897) took 1335 ms. Trace:
I20260709 13:11:18.267371 29152 rpcz_store.cc:276] 0709 13:11:16.931321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.689031 (+757710us) service_pool.cc:224] Handling call
0709 13:11:17.689437 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:17.707421 (+ 17984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:17.707477 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:17.718241 (+ 10764us) log.cc:844] Serialized 26 byte log entry
0709 13:11:18.106618 (+388377us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:18.106659 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.106727 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:18.266759 (+160032us) raft_consensus.cc:1685] finished
0709 13:11:18.266780 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:18.266925 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8938,"raft.queue_time_us":8810,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":10421,"thread_start_us":349725,"threads_started":1,"wal-append.queue_time_us":489278}
I20260709 13:11:18.269191 30888 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.270334 29710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 528) took 1870 ms. Trace:
I20260709 13:11:18.270721 29710 rpcz_store.cc:276] 0709 13:11:16.399561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.529884 (+130323us) service_pool.cc:224] Handling call
0709 13:11:16.693710 (+163826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:16.754124 (+ 60414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.270111 (+1515987us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54004e2350. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:18.270247 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356280704}
I20260709 13:11:18.269197 30937 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:18.251609 31098 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:18.273480 30701 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:18.273984 31098 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.275002 31098 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:18.230647 31042 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.182263 30585 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:18.249325 31061 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.700s	user 0.022s	sys 0.024s
I20260709 13:11:18.291318 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1904) took 1150 ms. Trace:
I20260709 13:11:18.281847 29150 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:18.250903 31054 raft_consensus.cc:2764] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:11:18.304718 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:11:18.307204 30990 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.006s
I20260709 13:11:18.307802 31041 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.309209 31001 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:18.310055 31001 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:18.310346 31001 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:18.311172 31001 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.312330 31042 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:18.314307 31108 consensus_meta.cc:287] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.899s	user 0.010s	sys 0.000s
I20260709 13:11:18.315415 30701 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.315913 30701 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.317942 30701 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:18.318074 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:18.318675 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:18.319311 29003 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:18.319373 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:18.245524 31094 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.320222 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:18.320703 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.321277 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:18.321519 29009 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:18.321986 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:18.322357 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:18.322435 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:18.322829 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:18.285768 31117 raft_consensus.cc:2764] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:11:18.279157 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:18.338757 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1125) took 1591 ms. Trace:
I20260709 13:11:18.338701 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:18.322635 29006 raft_consensus.cc:2368] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:18.339568 29155 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:18.339264 29151 rpcz_store.cc:276] 0709 13:11:16.747079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.747216 (+   137us) service_pool.cc:224] Handling call
0709 13:11:18.338557 (+1591341us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b5400398d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:18.338696 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291744128}
I20260709 13:11:18.340791 31001 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:18.341007 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:18.342191 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:18.342350 31095 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.342869 31095 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.344173 30585 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:18.344305 30990 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:18.344532 29083 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:18.339218 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60587.7us
I20260709 13:11:18.346876 29152 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:18.290995 31050 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.581s	user 0.006s	sys 0.004s
I20260709 13:11:18.347528 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:18.348317 29002 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:18.246825 31099 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.337710 30950 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.337607 30701 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:18.338204 31094 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:18.339568 31117 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:18.323119 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:18.363791 29010 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:18.350201 31050 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:18.344882 30990 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.355959 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1888) took 2388 ms. Trace:
I20260709 13:11:18.190727 31092 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:18.345472 29005 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:18.324137 29122 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:18.361156 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:18.362299 29006 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:18.321616 31118 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:18.376353 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.372321 29007 rpcz_store.cc:276] 0709 13:11:15.967617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.202218 (+234601us) service_pool.cc:224] Handling call
0709 13:11:18.203229 (+2001011us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:18.248492 (+ 45263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.355682 (+107190us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b540035d9d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:18.355833 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4577600384}
I20260709 13:11:18.344179 29155 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:18.365201 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:18.279241 31027 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.358702 31025 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.381695 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:18.381505 30761 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.384887 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:18.387187 28905 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52660 (request call id 246) took 12853 ms (client timeout 15000 ms). Trace:
I20260709 13:11:18.392402 30810 raft_consensus.cc:993] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:11:18.395543 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:18.433161 29152 raft_consensus.cc:2393] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:18.396522 29150 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:18.424772 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45038.9us
W20260709 13:11:18.425633 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 101578us
I20260709 13:11:18.292965 29154 rpcz_store.cc:276] 0709 13:11:17.140901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.913356 (+772455us) service_pool.cc:224] Handling call
0709 13:11:17.989164 (+ 75808us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:18.058232 (+ 69068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:18.058280 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:18.066832 (+  8552us) log.cc:844] Serialized 26 byte log entry
0709 13:11:18.217578 (+150746us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:18.217614 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.217676 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:18.291048 (+ 73372us) raft_consensus.cc:1685] finished
0709 13:11:18.291077 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:18.291247 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3609,"raft.queue_time_us":77773,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":33833,"thread_start_us":84139,"threads_started":1,"wal-append.queue_time_us":114171}
I20260709 13:11:18.285068 29157 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:18.363440 31117 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:18.374423 31002 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.377316 30937 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.379665 30958 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:18.470057 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:18.482693 30810 raft_consensus.cc:1081] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:11:18.474156 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:18.473353 29122 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:18.578279 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 105010us
I20260709 13:11:18.579152 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:18.475652 29151 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:18.365612 30950 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.583458 29122 leader_election.cc:400] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:18.577329 29150 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:18.475955 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:18.482136 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:18.586105 29080 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:18.470832 29010 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:18.567070 29078 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:11:18.568389 29079 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:18.563489 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:18.470894 30701 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:18.393061 29713 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:18.475557 31117 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:18.474828 31139 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:18.474179 30780 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.471210 31094 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:18.471833 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:18.575611 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:18.557991 31042 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:11:18.537544 31002 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.469013 31118 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.597930 29082 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:18.560334 30829 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:18.599041 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:18.610657 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:18.610998 29085 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:18.586575 31061 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:11:18.615407 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.472246 31025 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.583523 30990 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:18.572300 30778 consensus_queue.cc:1059] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:11:18.622354 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:18.586731 30718 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:18.606561 29077 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:18.613526 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:18.613788 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:18.615902 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.606988 31095 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.604596 29084 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:18.593441 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:18.623205 29712 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 2: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:18.625669 29085 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:18.633756 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:18.634428 29085 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:18.625923 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:18.627616 31002 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.603590 30780 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.586503 30937 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:18.432788 28905 rpcz_store.cc:269] 0709 13:11:05.537049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:05.563622 (+ 26573us) service_pool.cc:224] Handling call
0709 13:11:18.386941 (+12823319us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:12.844222 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:12.944795 (+100573us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:12.944829 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:11:12.944846 (+    17us) tablet.cc:662] Decoding operations
0709 13:11:12.951885 (+  7039us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:12.951946 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:12.951975 (+    29us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:11:12.998330 (+ 46355us) tablet.cc:701] Row locks acquired
0709 13:11:12.999905 (+  1575us) write_op.cc:260] PREPARE: finished
0709 13:11:13.041775 (+ 41870us) write_op.cc:270] Start()
0709 13:11:13.059659 (+ 17884us) write_op.cc:276] Timestamp: P: 1783602673014297 usec, L: 0
0709 13:11:13.070455 (+ 10796us) op_driver.cc:348] REPLICATION: starting
0709 13:11:13.155510 (+ 85055us) log.cc:844] Serialized 2074 byte log entry
0709 13:11:15.334467 (+2178957us) spinlock_profiling.cc:243] Waited 50.5 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:16.036693 (+702226us) op_driver.cc:464] REPLICATION: finished
0709 13:11:16.266237 (+229544us) write_op.cc:301] APPLY: starting
0709 13:11:16.288415 (+ 22178us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:16.399743 (+111328us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:16.419876 (+ 20133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:16.536793 (+116917us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:16.537093 (+   300us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:11:16.537169 (+    76us) write_op.cc:312] APPLY: finished
0709 13:11:16.566815 (+ 29646us) log.cc:844] Serialized 47 byte log entry
0709 13:11:16.583722 (+ 16907us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:16.583878 (+   156us) write_op.cc:454] Released schema lock
0709 13:11:16.583954 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:16.584476 (+   522us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:16.584505 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":555266,"reported_tablets":17,"spinlock_wait_cycles":979328,"child_traces":[["op",{"apply.queue_time_us":178411,"apply.run_cpu_time_us":3957,"apply.run_wall_time_us":358263,"num_ops":5,"prepare.queue_time_us":298211,"prepare.run_cpu_time_us":45993,"prepare.run_wall_time_us":502748,"raft.queue_time_us":322869,"raft.run_cpu_time_us":25241,"raft.run_wall_time_us":162962,"replication_time_us":2964656,"spinlock_wait_cycles":116241792,"thread_start_us":664120,"threads_started":4,"wal-append.queue_time_us":296479,"wal-append.run_cpu_time_us":41745,"wal-append.run_wall_time_us":4135652}]]}
I20260709 13:11:18.609308 31139 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:18.620411 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 149682us
I20260709 13:11:18.637303 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:18.654551 29085 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:18.657120 29085 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:18.638550 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45213.7us
I20260709 13:11:18.629698 31025 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.662048 29085 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:18.608496 31094 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:18.617038 30829 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.583422 30958 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.639230 31061 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:18.675988 29159 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:18.667330 29082 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:18.669829 30585 consensus_queue.cc:1059] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:18.679162 30761 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.671034 31099 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.674093 30937 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:18.676172 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:18.661695 30718 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:18.683259 30718 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:18.670883 30829 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.683676 30780 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:18.687521 31027 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.666991 31095 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.679948 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:11:18.692585 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 218612us
I20260709 13:11:18.695341 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:18.697311 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:18.696182 29150 raft_consensus.cc:2376] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5. Re-sending same reply.
I20260709 13:11:18.683702 30718 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:18.740501 29001 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:18.691558 31061 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:18.787789 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 128271us
W20260709 13:11:18.864558 29009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.025s	sys 0.018s
W20260709 13:11:18.865059 29009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.025s	sys 0.018s
I20260709 13:11:18.699849 31095 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:18.672657 30958 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:18.872264 30950 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:18.903486 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 113981us
I20260709 13:11:18.889901 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:18.904662 29085 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:11:18.925104 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 194024us
I20260709 13:11:18.903707 31004 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:18.913727 29711 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.022s	sys 0.034s
W20260709 13:11:19.015306 29711 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.022s	sys 0.034s
I20260709 13:11:19.018009 31064 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:19.026516 31152 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:11:19.066813 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 390443us
I20260709 13:11:18.842615 31139 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.068732 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:19.070194 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.072639 29150 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:19.014879 30637 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.011171 31108 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:11:19.009166 29077 raft_consensus.cc:2376] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5. Re-sending same reply.
I20260709 13:11:19.078220 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.082017 29159 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:19.009025 31025 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:11:19.008877 31099 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:11:19.084625 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.092103 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:19.092828 29159 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:19.087560 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:19.101831 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 532) took 1551 ms. Trace:
I20260709 13:11:19.104524 31064 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:19.105834 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 191474us
I20260709 13:11:19.106683 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:19.126556 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.193045 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.197566 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.195452 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:19.073781 30984 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.116731 29150 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:19.193823 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1891) took 2300 ms. Trace:
I20260709 13:11:19.194417 29156 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:11:19.195111 29084 consensus_meta.cc:287] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.009s
I20260709 13:11:19.194968 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.203922 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.202800 29153 rpcz_store.cc:276] 0709 13:11:16.893104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.383341 (+490237us) service_pool.cc:224] Handling call
0709 13:11:19.192814 (+1809473us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540041a0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:19.193768 (+   954us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332059904}
I20260709 13:11:19.202486 29159 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:19.219815 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.220760 29155 raft_consensus.cc:2393] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:19.225660 31117 consensus_meta.cc:287] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.623s	user 0.004s	sys 0.004s
I20260709 13:11:19.226069 31117 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:19.228642 31167 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:11:19.230167 31095 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:19.074285 30888 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:11:19.087724 31094 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:19.089267 31061 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.236711 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.237505 29158 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:19.238575 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:19.239488 29154 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:19.102303 29711 rpcz_store.cc:276] 0709 13:11:17.550551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.552005 (+  1454us) service_pool.cc:224] Handling call
0709 13:11:18.357786 (+805781us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:18.357982 (+   196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:18.358218 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:19.015867 (+657649us) log.cc:844] Serialized 14 byte log entry
0709 13:11:19.101435 (+ 85568us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:19.101497 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:19.101564 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.101636 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:19.101771 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":710189,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"spinlock_wait_cycles":7292288,"thread_start_us":742254,"threads_started":2}
W20260709 13:11:19.249181 31131 log.cc:927] Time spent T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 1.516s	user 0.007s	sys 0.019s
W20260709 13:11:19.250092 31157 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:11:19.023903 31004 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.253096 31004 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:19.082285 30637 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:19.257496 31098 consensus_meta.cc:287] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.982s	user 0.007s	sys 0.004s
W20260709 13:11:19.259500 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 64128.8us
I20260709 13:11:19.260848 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:19.262122 30922 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.263000 31098 raft_consensus.cc:2764] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:11:19.262648 30922 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:19.255895 30637 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:19.265412 30922 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:19.266072 29051 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:19.266760 31160 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:19.267236 29050 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:19.268466 31117 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:19.267776 31098 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:19.262646 31064 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:19.275959 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.300266 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 102422us
I20260709 13:11:19.301280 29682 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:19.339052 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 66137.4us
W20260709 13:11:19.338469 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 314918us
W20260709 13:11:19.344144 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 237509us
I20260709 13:11:19.345309 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:19.347185 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.347295 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 46141.3us
I20260709 13:11:19.348084 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:19.356674 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:19.353125 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:19.341286 28975 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:19.302170 31001 consensus_meta.cc:287] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.961s	user 0.003s	sys 0.008s
I20260709 13:11:19.363404 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:19.364377 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:19.316220 31092 consensus_meta.cc:287] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.741s	user 0.007s	sys 0.003s
I20260709 13:11:19.202363 30984 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.337420 30950 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:19.380116 30780 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:19.362135 31085 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.362509 30778 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:19.342744 31118 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:19.360867 30958 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:19.385524 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44358.4us
W20260709 13:11:19.388861 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 465168us
I20260709 13:11:19.374820 30950 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:19.362303 31064 consensus_queue.cc:1059] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:11:19.368014 31098 consensus_queue.cc:1059] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:11:19.370289 31117 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
W20260709 13:11:19.392786 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 487611us
W20260709 13:11:19.396749 29078 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.007s	sys 0.004s
I20260709 13:11:19.389880 28975 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:19.400250 29078 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.007s	sys 0.004s
I20260709 13:11:19.381181 31102 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:11:19.406514 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:19.407317 29082 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:19.395421 30950 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:19.395848 31118 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:19.424579 30585 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:19.426996 31082 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:19.427928 31149 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.533s	user 0.002s	sys 0.008s
I20260709 13:11:19.429713 31082 raft_consensus.cc:2764] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:11:19.430344 31082 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:19.430677 31082 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:19.431401 30678 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:19.432080 30678 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:19.431488 31082 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:19.434839 31082 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:19.433894 30678 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:19.436299 30678 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:19.436549 31153 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.544s	user 0.000s	sys 0.001s
I20260709 13:11:19.436776 30678 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:19.438060 29710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 323) took 1134 ms. Trace:
I20260709 13:11:19.438565 29710 rpcz_store.cc:276] 0709 13:11:18.304004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.319632 (+ 15628us) service_pool.cc:224] Handling call
0709 13:11:18.319987 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:18.320422 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:18.320452 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:18.321067 (+   615us) log.cc:844] Serialized 52 byte log entry
0709 13:11:18.619792 (+298725us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:18.624255 (+  4463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.624368 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:19.435964 (+811596us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:19.437584 (+  1620us) raft_consensus.cc:1685] finished
0709 13:11:19.437629 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:19.437979 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19798,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":6375808,"thread_start_us":287060,"threads_started":1,"wal-append.queue_time_us":370547}
I20260709 13:11:19.396417 30637 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:19.446825 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:19.395373 31170 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:19.447304 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:19.471239 29080 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:19.394217 31042 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:11:19.398168 31173 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:19.393313 30958 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:19.386884 31085 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.620667 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:19.625000 29082 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:19.419612 31177 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.442631 31156 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:19.625881 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:19.627851 29082 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:19.442407 31175 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:19.688187 29082 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:19.689567 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:19.690681 29082 raft_consensus.cc:2410] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:19.662400 29077 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:11:19.715945 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 313834us
W20260709 13:11:19.717654 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 273270us
W20260709 13:11:19.719017 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 373737us
I20260709 13:11:19.721402 29122 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:19.722256 29123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.010s	sys 0.026s
W20260709 13:11:19.722652 29123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.010s	sys 0.026s
I20260709 13:11:19.725560 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:19.734561 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:19.738602 29122 leader_election.cc:400] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5.
I20260709 13:11:19.751091 30675 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:19.751959 30675 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.814705 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1453) took 2140 ms. Trace:
I20260709 13:11:19.815371 29083 rpcz_store.cc:276] 0709 13:11:17.673869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.845693 (+171824us) service_pool.cc:224] Handling call
0709 13:11:18.344377 (+498684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:18.344904 (+   527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.814400 (+1469496us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b540034d850. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:19.814621 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146182528}
W20260709 13:11:19.753769 31161 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.518s	user 0.004s	sys 0.004s
W20260709 13:11:19.756230 31166 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.558s	user 0.003s	sys 0.000s
I20260709 13:11:19.758221 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1143) took 1420 ms. Trace:
I20260709 13:11:19.817937 29156 rpcz_store.cc:276] 0709 13:11:18.337420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.343936 (+  6516us) service_pool.cc:224] Handling call
0709 13:11:19.194186 (+850250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:19.203169 (+  8983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.757974 (+554805us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b540041a0d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:19.758147 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1954475776}
I20260709 13:11:19.762264 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 325) took 1456 ms. Trace:
I20260709 13:11:19.762312 31174 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:19.819633 31174 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.819404 29713 rpcz_store.cc:276] 0709 13:11:18.306082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.306467 (+   385us) service_pool.cc:224] Handling call
0709 13:11:18.306753 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:18.392941 (+ 86188us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:18.700812 (+307871us) log.cc:844] Serialized 14 byte log entry
0709 13:11:18.903595 (+202783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:18.903663 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:18.903726 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.755667 (+851941us) spinlock_profiling.cc:243] Waited 74.4 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:19.762058 (+  6391us) spinlock_profiling.cc:243] Waited 85.9 ms on lock 0x7b5400730fd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:19.762103 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:19.762212 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107069,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":9493,"spinlock_wait_cycles":368718464,"thread_start_us":202384,"threads_started":1,"wal-append.queue_time_us":373481}
I20260709 13:11:19.765249 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:19.767930 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:19.822503 29076 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:19.822990 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1040) took 1281 ms. Trace:
I20260709 13:11:19.823344 29076 rpcz_store.cc:276] 0709 13:11:18.541192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.553977 (+ 12785us) service_pool.cc:224] Handling call
0709 13:11:19.767749 (+1213772us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:19.822922 (+ 55173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25507072}
I20260709 13:11:19.772256 28908 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:19.776934 31107 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:19.826839 28908 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:19.782299 31079 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:19.828292 31079 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:19.829403 31079 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:19.784091 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:19.784996 31050 raft_consensus.cc:3055] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:19.831664 31050 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:19.832765 31050 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.834203 31050 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:19.787050 31095 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:19.836323 31095 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:19.837370 31095 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:19.839867 31095 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:19.789307 31158 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.541s	user 0.005s	sys 0.004s
I20260709 13:11:19.791633 31041 raft_consensus.cc:3055] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:19.845611 31041 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:19.793867 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.795485 31148 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.901s	user 0.002s	sys 0.000s
W20260709 13:11:19.799921 31151 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.909s	user 0.005s	sys 0.005s
W20260709 13:11:19.801743 31162 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.582s	user 0.001s	sys 0.009s
I20260709 13:11:19.851620 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 219) took 1508 ms. Trace:
I20260709 13:11:19.852212 29009 rpcz_store.cc:276] 0709 13:11:18.342838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.345787 (+  2949us) service_pool.cc:224] Handling call
0709 13:11:18.357213 (+ 11426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:18.357480 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:18.357515 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:18.358038 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:11:18.865500 (+507462us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:18.865557 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.865649 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:19.801079 (+935430us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:19.851240 (+ 50161us) raft_consensus.cc:1685] finished
0709 13:11:19.851270 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:19.851554 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113638784,"thread_start_us":506980,"threads_started":1,"wal-append.queue_time_us":731312}
I20260709 13:11:19.655711 31173 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:11:19.713899 31175 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:19.899051 29680 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.021s	sys 0.037s
W20260709 13:11:19.899497 29680 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.021s	sys 0.037s
W20260709 13:11:19.886170 30780 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.022s	sys 0.054s
I20260709 13:11:19.858619 31170 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:19.860162 31171 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:11:19.895679 30718 consensus_meta.cc:287] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.110s	user 0.011s	sys 0.000s
W20260709 13:11:19.906313 29125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.016s	sys 0.018s
W20260709 13:11:19.912967 29125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.016s	sys 0.018s
I20260709 13:11:19.912822 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:19.913327 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 547745us
W20260709 13:11:19.906574 29004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.026s	sys 0.053s
I20260709 13:11:19.659538 31177 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:19.911963 29151 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:19.913476 29159 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:19.914319 29004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.026s	sys 0.053s
I20260709 13:11:19.910228 29122 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:19.905247 31160 consensus_meta.cc:287] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.005s
W20260709 13:11:19.903402 31092 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.013s	sys 0.045s
W20260709 13:11:19.921757 29078 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.012s	sys 0.001s
W20260709 13:11:19.922897 29124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.008s	sys 0.006s
I20260709 13:11:19.926095 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.932248 29156 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:19.932439 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.902813 30990 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.933056 29157 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 2.
W20260709 13:11:19.906961 30780 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.023s	sys 0.054s
I20260709 13:11:19.929330 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.934170 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:19.902737 30984 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:19.934585 29158 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:19.904505 31061 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:19.930796 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:19.908233 31169 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.001878 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:19.943076 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:20.006865 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63860.5us
W20260709 13:11:20.053910 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 143740us
I20260709 13:11:20.055078 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:20.061120 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53135.7us
I20260709 13:11:20.064340 29125 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:19.935746 31099 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.003s	sys 0.012s
I20260709 13:11:19.913717 30637 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.928828 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:19.910038 28908 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 2 to 3, term changed from 1 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66), ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:19.943132 29159 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:19.964818 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:19.933329 30990 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:20.103139 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:19.935986 29124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.008s	sys 0.007s
I20260709 13:11:19.936607 30479 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:19.931453 31092 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.013s	sys 0.045s
I20260709 13:11:20.142951 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:20.146183 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:20.147291 29159 raft_consensus.cc:2410] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:20.000893 31176 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.003052 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:20.058737 31191 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.053622 31171 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:19.935159 31160 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:20.112213 31099 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.003s	sys 0.013s
W20260709 13:11:19.931600 29078 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.012s	sys 0.001s
W20260709 13:11:19.927337 30829 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.000s	sys 0.009s
I20260709 13:11:19.919523 30718 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:19.942902 31184 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.138746 30716 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.150731 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:20.150933 30479 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:20.152395 31180 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 13:11:20.161891 31179 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 13:11:20.162266 29710 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:20.141595 30637 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.065955 31173 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:20.167014 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:20.109218 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:20.192598 29154 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:20.181449 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:20.234895 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:20.246031 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:20.058813 31169 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.264308 29158 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:20.260131 31170 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:20.264037 31171 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.270258 29709 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 2.
W20260709 13:11:20.254709 30829 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.000s	sys 0.018s
I20260709 13:11:20.181250 31168 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:20.180933 30780 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:20.135859 29051 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:20.288671 28908 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:20.175505 31185 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:20.293527 31095 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:20.283526 31082 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.848s	user 0.008s	sys 0.002s
I20260709 13:11:20.180009 30984 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.175680 31175 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:20.182392 31176 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.285519 31160 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:20.267352 30958 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:20.339092 31170 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:20.364269 29010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.021s	sys 0.035s
W20260709 13:11:20.379140 29683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.018s	sys 0.040s
W20260709 13:11:20.313611 31178 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:11:20.064512 31061 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.266219 30716 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.302013 31085 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:20.399596 31050 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
I20260709 13:11:20.323977 30637 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:20.263726 29711 raft_consensus.cc:2955] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:20.341554 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1044) took 1795 ms. Trace:
I20260709 13:11:20.340754 31193 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:11:20.399317 28975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.028s	sys 0.079s
W20260709 13:11:20.495345 28975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.029s	sys 0.079s
I20260709 13:11:20.291199 31099 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:20.408541 28976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.025s	sys 0.035s
W20260709 13:11:20.505599 28976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.026s	sys 0.035s
W20260709 13:11:20.412792 29683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.018s	sys 0.040s
I20260709 13:11:20.506641 29683 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:20.508005 29683 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:20.412379 29010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.021s	sys 0.035s
I20260709 13:11:20.510303 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:20.299258 30675 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:20.408800 31025 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.211s	user 0.005s	sys 0.006s
W20260709 13:11:20.422376 29081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.010s	sys 0.001s
W20260709 13:11:20.533385 29050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.035s	sys 0.058s
W20260709 13:11:20.535585 29050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.035s	sys 0.058s
W20260709 13:11:20.537035 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 1.16334e+06us
W20260709 13:11:20.489434 29048 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.015s	sys 0.084s
W20260709 13:11:20.540896 29048 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.015s	sys 0.084s
I20260709 13:11:20.425354 31183 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:20.409552 30780 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:20.500438 31193 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:20.511320 29682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.140s	user 0.024s	sys 0.055s
I20260709 13:11:20.421126 30958 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.298790 31173 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:20.514272 29683 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:20.533051 29081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.010s	sys 0.001s
W20260709 13:11:20.298979 30678 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.857s	user 0.005s	sys 0.005s
W20260709 13:11:20.393199 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 257433us
I20260709 13:11:20.572117 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:20.488396 30479 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:20.393168 31095 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:20.533865 30888 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:20.194334 31191 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.593075 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1046) took 2046 ms. Trace:
I20260709 13:11:20.593451 29078 rpcz_store.cc:276] 0709 13:11:18.547070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.589043 (+ 41973us) service_pool.cc:224] Handling call
0709 13:11:18.606039 (+ 16996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:18.674324 (+ 68285us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:18.828240 (+153916us) log.cc:844] Serialized 14 byte log entry
0709 13:11:20.249820 (+1421580us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:20.249874 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:20.249938 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:20.592839 (+342901us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b54000c8fd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:20.592886 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:20.593013 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12397,"prepare.queue_time_us":16896,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":900409,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":36749,"spinlock_wait_cycles":154983296,"thread_start_us":1562069,"threads_started":3,"wal-append.queue_time_us":915788}
I20260709 13:11:20.595018 31191 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:20.595649 31108 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:20.595818 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1894) took 4150 ms. Trace:
I20260709 13:11:20.596076 31108 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:20.425187 31186 consensus_queue.cc:1059] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.147s
I20260709 13:11:20.530835 30504 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:11:20.596896 31108 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:20.498911 29079 rpcz_store.cc:276] 0709 13:11:18.546530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.588640 (+ 42110us) service_pool.cc:224] Handling call
0709 13:11:18.604191 (+ 15551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:18.604640 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:11:18.604672 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:18.605196 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:11:19.271193 (+665997us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b18001066c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:11:19.626927 (+355734us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:19.626970 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.627053 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:20.341232 (+714179us) raft_consensus.cc:1685] finished
0709 13:11:20.341251 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:20.341479 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":667,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":6602,"spinlock_wait_cycles":8674688,"thread_start_us":355543,"threads_started":1,"wal-append.queue_time_us":1296923}
I20260709 13:11:20.598244 31108 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:20.582814 30778 consensus_meta.cc:287] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.196s	user 0.008s	sys 0.000s
I20260709 13:11:20.598872 30778 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:20.601022 31207 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.601382 30718 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:20.601558 31207 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:20.542423 28973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.008s	sys 0.002s
I20260709 13:11:20.505498 31099 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:20.534035 31025 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.006s	sys 0.006s
I20260709 13:11:20.549330 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:20.439473 28974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.005s	sys 0.006s
I20260709 13:11:20.553861 29082 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:11:20.419420 28908 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:20.513418 31187 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:20.421458 30984 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.269294 31184 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:20.582814 30950 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.015s	user 0.007s	sys 0.000s
I20260709 13:11:20.537897 29708 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:20.604023 31207 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:20.568858 31183 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.586858 31132 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:20.587795 30921 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:20.589134 30716 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:20.589531 31192 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:20.591235 31107 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:20.614851 30888 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:20.592468 31176 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:20.544119 31197 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.596276 29004 rpcz_store.cc:276] 0709 13:11:16.445262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.598337 (+153075us) service_pool.cc:224] Handling call
0709 13:11:16.598708 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:16.634656 (+ 35948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:16.634701 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:17.515168 (+880467us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b18000cf160. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a
0709 13:11:19.271917 (+1756749us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b2c00000f48. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:19.337298 (+ 65381us) log.cc:844] Serialized 26 byte log entry
0709 13:11:19.916565 (+579267us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:19.916622 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.916685 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:20.595613 (+678928us) raft_consensus.cc:1685] finished
0709 13:11:20.595632 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:20.595760 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6707,"spinlock_wait_cycles":7927808,"thread_start_us":536120,"threads_started":1,"wal-append.queue_time_us":1173880}
I20260709 13:11:20.533959 30675 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:20.392903 31168 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:20.543731 29682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.173s	user 0.024s	sys 0.056s
W20260709 13:11:20.539791 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 475458us
I20260709 13:11:20.341503 31171 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:20.542269 31139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.304s	user 0.004s	sys 0.009s
I20260709 13:11:20.414300 31160 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:20.541949 31098 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.005s	sys 0.007s
I20260709 13:11:20.610062 31099 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:20.610630 29080 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
W20260709 13:11:20.611084 28974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.005s	sys 0.006s
W20260709 13:11:20.604331 28973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.008s	sys 0.002s
I20260709 13:11:20.613236 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:20.616534 29084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.000s	user 0.010s	sys 0.008s
W20260709 13:11:20.617252 29122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.002s	sys 0.009s
I20260709 13:11:20.616556 28908 catalog_manager.cc:5697] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 1 to 2, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 2 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:20.620867 31169 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:20.584808 31177 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:20.584507 31095 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:20.623842 29156 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:20.732219 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:20.639106 29151 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:20.650182 29713 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:20.632929 31000 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.031s	sys 0.068s
I20260709 13:11:20.618423 31191 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:20.576431 30958 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.622185 31196 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:20.664508 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 1.3113e+06us
W20260709 13:11:20.663687 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 109542us
I20260709 13:11:20.665117 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:20.689083 31139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.451s	user 0.005s	sys 0.009s
I20260709 13:11:20.663198 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:20.665515 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 167147us
I20260709 13:11:20.696344 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:20.706718 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:20.610455 31170 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:20.711635 31201 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:11:20.715204 31094 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:20.717411 31174 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:20.721275 31188 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:11:20.723109 30990 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:20.725934 31079 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:20.638687 31132 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:20.733548 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 161510us
W20260709 13:11:20.649729 31156 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.989s	user 0.008s	sys 0.003s
I20260709 13:11:20.608428 30678 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:20.627955 31025 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:11:20.649778 31107 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:20.616349 31064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.215s	user 0.000s	sys 0.018s
I20260709 13:11:20.628242 30950 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:20.638635 30780 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:20.977427 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 312366us
I20260709 13:11:20.978644 29159 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
W20260709 13:11:20.769027 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 227790us
W20260709 13:11:20.982528 29084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.366s	user 0.010s	sys 0.012s
I20260709 13:11:20.984684 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:20.980499 29122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.006s	sys 0.009s
I20260709 13:11:20.996466 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:20.654752 31207 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:21.006191 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1045) took 2436 ms. Trace:
I20260709 13:11:20.656960 31193 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:20.983157 31098 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.564s	user 0.021s	sys 0.033s
I20260709 13:11:21.047770 29085 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
W20260709 13:11:20.998854 31064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.600s	user 0.000s	sys 0.022s
W20260709 13:11:20.763862 31206 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.086s	user 0.004s	sys 0.000s
I20260709 13:11:20.658429 31195 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:20.978138 31000 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.231s	user 0.065s	sys 0.122s
I20260709 13:11:21.012674 29050 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:21.013319 29683 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:21.013489 29157 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:11:20.981778 31094 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:21.014408 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.002480 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.065755 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.066649 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.068411 29003 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:20.866460 31171 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.006978 29084 rpcz_store.cc:276] 0709 13:11:18.546794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.588661 (+ 41867us) service_pool.cc:224] Handling call
0709 13:11:18.604298 (+ 15637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:18.604496 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:19.203225 (+598729us) log.cc:844] Serialized 14 byte log entry
0709 13:11:20.983111 (+1779886us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:20.983184 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:20.983258 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:20.983347 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:20.983488 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23110,"raft.queue_time_us":941394,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":21449,"thread_start_us":1779460,"threads_started":2,"wal-append.queue_time_us":705004}
I20260709 13:11:21.004333 31156 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:20.978677 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:20.661824 31210 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:21.012482 31041 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.688926 31160 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.764662 30888 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.809473 31173 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:20.811095 31184 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.029165 31207 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:20.838191 31168 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.031024 31187 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:21.044902 31183 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:20.886590 30718 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.915294 31186 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.807623 31203 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.831875 31092 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.678831 31197 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.001227 31198 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:21.003350 31001 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.999756 31118 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:21.000986 31214 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:21.033638 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.137878 29079 raft_consensus.cc:2410] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:21.034194 31225 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.035243 31107 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.036676 30984 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.039109 31190 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:20.658780 31192 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.005649 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1156) took 1075 ms. Trace:
I20260709 13:11:20.858685 31144 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.044838 30950 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:21.056346 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 47179.8us
I20260709 13:11:21.059782 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1052) took 1614 ms. Trace:
I20260709 13:11:21.186759 29085 rpcz_store.cc:276] 0709 13:11:19.445153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.446577 (+  1424us) service_pool.cc:224] Handling call
0709 13:11:21.059561 (+1612984us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000465d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:21.059706 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3632575744}
I20260709 13:11:20.813400 31215 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:20.822234 30922 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:20.998436 31132 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:21.064247 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51001.9us
W20260709 13:11:21.064313 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51727.3us
I20260709 13:11:21.079787 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.191761 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.080910 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.192483 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:21.083348 29002 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:21.193868 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:21.194202 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.194770 29002 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:21.195575 29050 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:21.015385 31177 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:20.764835 31209 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.110769 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.097533 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.201104 29007 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:21.201333 29159 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
W20260709 13:11:21.095752 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 432628us
I20260709 13:11:21.202833 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:20.733237 30675 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.210433 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.135913 30479 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:11:21.117425 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.111171 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.001665 30716 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:11:21.119099 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.004096 31191 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.090342 31061 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:21.132520 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 153902us
I20260709 13:11:21.090612 30921 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:21.110689 30778 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:21.080479 30972 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.128371 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.082536 31139 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:21.136986 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.231487 29150 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:21.118412 31041 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:21.141736 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 227) took 1746 ms. Trace:
I20260709 13:11:21.111639 31184 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:21.003875 31196 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.143780 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1038) took 2603 ms. Trace:
I20260709 13:11:21.090169 31095 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:11:21.118515 31210 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:21.177618 31001 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.121665 31212 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.186571 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.231895 31107 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:21.186619 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:21.259469 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 72920.7us
I20260709 13:11:21.260221 28975 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:21.261353 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:21.179203 31225 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.263785 31225 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.125128 30718 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.265714 30718 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.266331 31204 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:21.266776 31204 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:21.267123 31204 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:21.088807 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.110809 31221 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:11:21.179729 31094 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.221614 31177 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.245878 31102 raft_consensus.cc:3055] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:21.192432 29006 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:21.289413 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.290107 29006 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:21.136992 31193 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:21.117235 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:21.211416 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 226809us
I20260709 13:11:21.136746 31205 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.209383 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:21.296417 31205 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.298502 31205 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.299052 31205 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:21.299386 31205 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.299705 31205 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:21.245792 30778 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.270733 31229 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:21.301810 30778 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.247857 30972 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.214147 28976 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:21.140475 31118 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:21.220434 29004 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 2.
I20260709 13:11:21.128629 31220 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.224498 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:21.183808 31176 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.184508 31190 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.183476 31171 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:21.178810 31213 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:21.196666 30922 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.196977 31215 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:21.128446 31186 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.212613 31098 raft_consensus.cc:2764] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:11:21.320264 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 110984us
I20260709 13:11:21.223452 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:21.321228 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:21.321385 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 98013.5us
I20260709 13:11:21.196496 31216 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.135264 31092 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.285317 30678 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.242034 29010 rpcz_store.cc:276] 0709 13:11:19.395231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.395369 (+   138us) service_pool.cc:224] Handling call
0709 13:11:19.395740 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:19.400946 (+  5206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:19.400983 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:19.412659 (+ 11676us) log.cc:844] Serialized 26 byte log entry
0709 13:11:20.509638 (+1096979us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:20.509677 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:20.509747 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:20.710890 (+201143us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:21.133216 (+422326us) raft_consensus.cc:1685] finished
0709 13:11:21.133244 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:21.141650 (+  8406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4772,"raft.queue_time_us":37583,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":6340352,"thread_start_us":854373,"threads_started":1,"wal-append.queue_time_us":897322}
I20260709 13:11:21.332471 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.184747 29151 rpcz_store.cc:276] 0709 13:11:19.930401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.930541 (+   140us) service_pool.cc:224] Handling call
0709 13:11:21.005455 (+1074914us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54000951d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:21.005595 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817192832}
I20260709 13:11:21.244415 29682 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:21.333300 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 88980.3us
I20260709 13:11:21.243868 29002 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:21.244765 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.134719 31203 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.245044 29081 rpcz_store.cc:276] 0709 13:11:18.540584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.540983 (+   399us) service_pool.cc:224] Handling call
0709 13:11:18.541288 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:18.541646 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:18.541673 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:19.264570 (+722897us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b18000127c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a
0709 13:11:19.769126 (+504556us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b2c0004c3c8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:19.769742 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:11:20.554973 (+785231us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:20.555030 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:20.555123 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:21.143502 (+588379us) raft_consensus.cc:1685] finished
0709 13:11:21.143534 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:21.143703 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12159,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":2426,"spinlock_wait_cycles":63040640,"thread_start_us":784781,"threads_started":1,"wal-append.queue_time_us":853665}
I20260709 13:11:21.141081 31198 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:21.189067 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.215492 31175 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:11:21.268666 29153 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:21.179124 31214 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.077234 31195 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.221839 31209 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.241232 31224 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:21.131083 31027 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:21.241601 31139 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.331776 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.206300 31144 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:21.314546 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.316143 29003 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:21.317888 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 93466.6us
I20260709 13:11:21.248239 31200 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:11:21.249297 31184 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.318665 29004 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:21.345516 30972 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.327834 30718 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:21.338380 31233 raft_consensus.cc:2764] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:11:21.328320 31210 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:21.330341 31092 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.245827 30921 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.288800 31102 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:21.360684 31139 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:21.331176 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 65955.9us
I20260709 13:11:21.334669 29007 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:21.312354 31173 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:21.312602 31118 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:21.338563 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44669.2us
I20260709 13:11:21.344312 31238 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:21.315011 31190 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:21.317806 30922 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:21.324399 31223 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:21.325415 31094 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:21.356766 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:21.307090 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 93015.1us
I20260709 13:11:21.359750 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.337515 31207 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.358100 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:21.340265 30984 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:21.322642 31213 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.418701 31195 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.365747 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:21.349953 29711 consensus_meta.cc:287] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.728s	user 0.007s	sys 0.003s
I20260709 13:11:21.317175 31220 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.363608 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:21.367727 31144 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.216060 31025 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:21.324353 31216 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.373116 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.322150 31171 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.317394 31176 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.376003 29005 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:21.377805 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.352562 31224 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:21.269204 31212 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.407121 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.353258 31000 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:11:21.472575 28975 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:21.405651 29078 raft_consensus.cc:2410] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:21.405679 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:21.377538 30718 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.407138 29084 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:21.351969 31215 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:21.413287 31235 log.cc:927] Time spent T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:11:21.413957 31231 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:11:21.418797 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 195307us
W20260709 13:11:21.488652 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 91606.4us
I20260709 13:11:21.371439 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.491997 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.492498 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.446715 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.493168 29717 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:21.493474 29007 raft_consensus.cc:2410] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:21.392894 29713 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:21.472853 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 109310us
W20260709 13:11:21.483301 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 77698.7us
I20260709 13:11:21.495198 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:21.496315 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:21.445856 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:21.497426 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.485581 29010 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:21.480469 31192 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.446892 31197 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.406788 31223 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:21.492594 29004 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:21.415794 31099 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.501806 29154 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:21.499145 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.501848 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:21.499070 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:21.551090 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.475457 31224 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:21.554298 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.479700 31001 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.472378 31191 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:21.558041 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 56247.4us
I20260709 13:11:21.552862 29715 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:21.490000 31238 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:21.559758 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.487315 31168 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:21.556387 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.556566 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.487015 30718 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:21.487287 31229 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.580545 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.555016 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:21.531719 30972 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:21.550155 31079 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.547s	user 0.000s	sys 0.010s
I20260709 13:11:21.559538 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:21.563074 29083 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:21.561333 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:21.575369 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:21.531005 31187 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:21.499235 31025 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:21.486914 31213 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.583194 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.487802 29155 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:21.551149 31223 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:21.587327 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:21.588624 29004 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:21.563764 31225 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.589004 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.563961 31197 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.590500 28975 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:21.593354 29159 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:21.593734 29081 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:21.591418 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.594887 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.490880 31160 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.564378 31168 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.556520 30922 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.589500 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.563675 31001 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:21.531068 31144 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:11:21.594216 31233 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.577481 31177 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.560261 31207 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:11:21.602789 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.617712 28976 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:21.484498 31102 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:21.659159 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.617416 31183 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.618968 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:21.617069 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.554805 31195 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 13:11:21.602342 29152 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.014s	sys 0.014s
W20260709 13:11:21.668435 29152 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.015s	sys 0.014s
I20260709 13:11:21.618579 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:21.634575 31194 raft_consensus.cc:3055] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:21.643373 30990 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.646315 29708 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:21.593818 30678 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:21.596374 31027 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:21.609783 29083 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:21.594293 31241 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:21.608518 31079 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.607148 31169 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:21.656222 29150 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:21.708699 31216 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.711059 29150 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:21.659376 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:21.665272 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.620808 31168 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:21.669927 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.590483 31187 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:21.670142 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51646.5us
I20260709 13:11:21.730867 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:21.647727 30958 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.613380 31186 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.616555 31213 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.626652 31001 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.627532 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.694015 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:21.700407 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80257.7us
I20260709 13:11:21.705608 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.658552 31238 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:21.651012 31196 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.666533 31177 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:21.712023 29150 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:21.717192 30921 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.718042 31099 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.622180 31197 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.720491 31171 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.674417 31194 raft_consensus.cc:740] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:21.675554 31203 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.675319 31184 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.621932 31225 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.681015 31192 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:21.720501 31242 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 13:11:21.724771 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 65453.3us
I20260709 13:11:21.725564 31144 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:21.725793 31245 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:11:21.730466 28976 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:21.697737 31027 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.743660 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:21.745656 31132 consensus_meta.cc:287] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.001s
I20260709 13:11:21.660758 31233 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.748557 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 534) took 1837 ms. Trace:
I20260709 13:11:21.748833 31118 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.721050 31211 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:21.842614 31132 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.845993 31211 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:21.750792 30958 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:21.755453 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:21.847534 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 92153.8us
I20260709 13:11:21.590257 31229 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.758992 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:21.736610 31187 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:21.682364 31183 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.702236 31209 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.733892 31193 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.806315 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:21.697299 30990 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.734120 31216 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:21.752887 31004 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:21.801112 31107 consensus_meta.cc:287] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.001s
I20260709 13:11:21.712766 31079 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.754264 31001 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:21.816893 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 85446.8us
I20260709 13:11:21.824471 31251 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:21.828235 29003 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
W20260709 13:11:21.831162 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 240700us
I20260709 13:11:21.800001 31025 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:21.769501 29155 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:21.835246 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:21.894467 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 59282.8us
I20260709 13:11:21.836262 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.836596 29081 raft_consensus.cc:2410] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:11:21.817533 31108 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.217s	user 0.012s	sys 0.002s
I20260709 13:11:21.810137 31176 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.804651 29077 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:21.757009 31186 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:11:21.712834 31241 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:21.842758 31246 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:11:21.753958 31213 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.846411 31211 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:21.855384 29158 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 2.
I20260709 13:11:21.853403 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:21.843106 31248 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:21.838572 31027 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:21.820395 31223 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:21.948099 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:21.912868 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:21.861907 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 131508us
I20260709 13:11:21.964706 29001 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:11:21.965217 29001 pending_rounds.cc:85] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 1
I20260709 13:11:21.965487 29001 pending_rounds.cc:107] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:11:21.965780 29001 raft_consensus.cc:2889] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:21.846899 31177 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:21.848238 31220 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.850364 30958 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.852377 31144 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:21.975263 31144 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.864579 31210 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:21.866365 31192 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:21.922008 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.897217 29152 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:21.914565 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:21.917289 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 341942us
W20260709 13:11:21.958894 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 105576us
I20260709 13:11:21.866609 31212 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.924240 31171 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:21.896260 31160 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.947556 31209 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.960736 31197 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.961030 31118 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:21.961246 30990 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:21.961843 31132 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:21.867103 31193 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:21.974558 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.196s (SENT) [suppressed 19 similar messages]
I20260709 13:11:21.873728 29084 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:21.976929 29078 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:21.976881 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.034355 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:21.923779 31237 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:11:21.985708 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:22.027431 31041 consensus_meta.cc:287] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.782s	user 0.010s	sys 0.000s
I20260709 13:11:21.973496 31176 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:22.029098 31079 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.002359 29050 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:21.866466 31216 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:21.962725 31184 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:11:21.975886 30958 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:21.934407 31251 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.036496 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.058689 29157 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:21.984736 31144 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:21.920513 31255 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:22.036233 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 130539us
W20260709 13:11:22.063344 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61066.1us
I20260709 13:11:22.101503 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.102192 29081 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:22.066774 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:22.105336 29083 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:21.990396 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.106079 29150 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:22.030536 31171 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:22.060745 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 86232.2us
I20260709 13:11:22.030015 29709 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:22.063117 31118 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:22.096768 31197 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:21.864758 31194 consensus_queue.cc:260] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.097303 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.113862 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.060738 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.115291 31194 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:22.115231 31263 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:11:22.115994 29077 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:22.116830 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.086143 31209 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.117710 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:22.029112 31183 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.120738 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:22.059698 31176 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:22.064030 31156 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:22.124625 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.125403 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.136830 29150 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:22.109185 31171 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:22.130738 29081 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:22.135046 29713 raft_consensus.cc:2368] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:22.111639 31118 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.114085 31160 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:22.117218 31197 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.085628 30718 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.123142 31248 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.026762 31177 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.150753 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:22.136056 31236 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.307s	user 0.001s	sys 0.001s
I20260709 13:11:22.151437 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.127942 31117 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:22.139616 31176 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:22.145756 31118 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:22.161763 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 250) took 1148 ms. Trace:
I20260709 13:11:22.170866 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.172014 29083 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:22.151279 31197 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:22.167043 29151 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:22.084245 31025 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.164533 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:22.121315 31247 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.105s
I20260709 13:11:22.175534 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.170475 29082 rpcz_store.cc:276] 0709 13:11:21.013201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.032799 (+ 19598us) service_pool.cc:224] Handling call
0709 13:11:21.033150 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:21.197269 (+164119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:21.197300 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:21.211727 (+ 14427us) log.cc:844] Serialized 26 byte log entry
0709 13:11:21.315804 (+104077us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:21.315857 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:21.315959 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:22.161601 (+845642us) raft_consensus.cc:1685] finished
0709 13:11:22.161618 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:22.161717 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4851,"raft.queue_time_us":47485,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":25824,"thread_start_us":221399,"threads_started":2,"wal-append.queue_time_us":220034}
I20260709 13:11:22.171540 31117 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.176151 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.148885 31212 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:22.185058 29151 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:22.181684 29081 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:22.182014 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.183766 31233 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.186733 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.188743 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.189164 29082 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:22.163692 31225 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.188536 29077 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:22.205592 29151 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:22.177033 31251 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.205592 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:21.843124 29711 rpcz_store.cc:276] 0709 13:11:19.911504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.924425 (+ 12921us) service_pool.cc:224] Handling call
0709 13:11:20.139578 (+215153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:20.192789 (+ 53211us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:21.424881 (+1232092us) log.cc:844] Serialized 14 byte log entry
0709 13:11:21.747940 (+323059us) log.cc:844] Serialized 14 byte log entry
0709 13:11:21.748188 (+   248us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:21.748238 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:21.748306 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:21.748379 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:21.748500 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10158,"prepare.queue_time_us":243691,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":1592,"raft.queue_time_us":46059,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":8303,"thread_start_us":243107,"threads_started":2,"wal-append.queue_time_us":182484}
I20260709 13:11:22.146603 31156 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.211944 29157 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
W20260709 13:11:22.184506 31204 consensus_meta.cc:287] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.917s	user 0.001s	sys 0.011s
I20260709 13:11:22.213806 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.120945 31255 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.164178 31210 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.173995 31248 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.351171 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 209) took 1363 ms. Trace:
I20260709 13:11:22.221414 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.317881 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:22.359788 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.289080 29008 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:11:22.268963 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.223309 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.322773 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.360498 29158 rpcz_store.cc:276] 0709 13:11:20.988260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.002138 (+ 13878us) service_pool.cc:224] Handling call
0709 13:11:22.350993 (+1348855us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54004010d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:22.351108 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1817752448}
I20260709 13:11:22.213948 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.361655 29085 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:22.361871 29079 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:22.362008 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:22.360378 29151 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:22.307720 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:22.340027 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.363793 29151 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:22.364341 29078 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:22.317245 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.362586 29158 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:22.366514 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 48762.4us
I20260709 13:11:22.367372 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:22.365170 29081 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:22.368850 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.369585 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:22.370100 29079 raft_consensus.cc:2368] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:22.365556 29010 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:22.371199 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.371822 29085 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:22.372721 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.374243 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.191414 31241 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.200815 31213 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:22.227659 31094 consensus_meta.cc:287] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.819s	user 0.003s	sys 0.004s
I20260709 13:11:22.380380 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.376693 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.381748 29158 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:22.382042 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.382786 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:22.382898 29081 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
W20260709 13:11:22.278885 31267 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:11:22.339672 31205 consensus_meta.cc:287] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.021s	user 0.005s	sys 0.010s
I20260709 13:11:22.377475 29085 raft_consensus.cc:2410] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:22.377619 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:22.294129 31224 consensus_meta.cc:287] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.731s	user 0.006s	sys 0.003s
I20260709 13:11:22.204089 31108 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.384321 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.223834 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.278692 30718 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.191167 31212 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.320878 31193 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.268929 30780 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:22.327437 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.228753 31117 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.387458 30922 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.389436 31027 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.260035 31144 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.392102 31263 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:22.393363 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 215) took 1367 ms. Trace:
I20260709 13:11:22.393942 31118 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:22.396938 31190 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.971s	user 0.010s	sys 0.000s
W20260709 13:11:22.396938 31139 consensus_meta.cc:287] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.989s	user 0.009s	sys 0.001s
I20260709 13:11:22.397269 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 235) took 1267 ms. Trace:
I20260709 13:11:22.296180 31223 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.403092 29085 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:11:22.508802 29159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1928) took 1150 ms (client timeout 1516 ms). Trace:
I20260709 13:11:22.509742 29085 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:11:22.509511 29159 rpcz_store.cc:269] 0709 13:11:21.358075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.359543 (+  1468us) service_pool.cc:224] Handling call
0709 13:11:22.489514 (+1129971us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54002b66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:22.508724 (+ 19210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431739264}
I20260709 13:11:22.510723 29085 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:22.513211 29156 raft_consensus.cc:2368] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
W20260709 13:11:22.514415 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 410622us
I20260709 13:11:22.404218 29078 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:22.376255 31249 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:22.399920 31168 consensus_meta.cc:287] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.657s	user 0.002s	sys 0.003s
I20260709 13:11:22.409917 29081 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:11:22.482656 29083 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:22.409180 29710 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:22.519074 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:22.519860 29150 raft_consensus.cc:2368] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:22.520473 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:22.294200 31171 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.367797 31204 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.489898 29153 rpcz_store.cc:276] 0709 13:11:21.025637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.036304 (+ 10667us) service_pool.cc:224] Handling call
0709 13:11:21.036595 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:21.340667 (+304072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:22.393135 (+1052468us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54003ed5d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:22.393307 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532884736}
I20260709 13:11:22.505968 29004 rpcz_store.cc:276] 0709 13:11:21.129997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.586311 (+456314us) service_pool.cc:224] Handling call
0709 13:11:21.586591 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:21.604130 (+ 17539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:22.396961 (+792831us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b5400026054. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:22.397192 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4253056}
I20260709 13:11:22.481223 31160 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.480986 31095 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:11:22.519203 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.536875 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:22.487715 31229 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.405331 31224 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.540832 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:22.542361 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:22.513752 31267 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:11:22.546556 29077 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:22.516165 31249 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.403900 31205 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.514426 31263 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:22.517845 30718 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:11:22.538672 31183 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:22.549448 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.539740 31095 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:22.562549 29004 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:22.538241 31079 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:22.533761 31209 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.537668 31216 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.408583 31108 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.508713 31190 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.533439 31251 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:11:22.542791 31247 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:22.541690 29710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49414 (request call id 365) took 1044 ms. Trace:
I20260709 13:11:22.564682 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.577247 29125 leader_election.cc:400] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:22.551417 31249 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:22.547441 31266 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:11:22.564985 30984 consensus_meta.cc:287] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.145s	user 0.009s	sys 0.003s
I20260709 13:11:22.535151 31168 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:22.569552 29742 heartbeater.cc:646] Failed to heartbeat to 127.27.63.126:43467 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.27.63.126:43467 timed out after 15.000s (SENT)
I20260709 13:11:22.569869 31118 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.580499 31266 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.543985 31160 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.596328 31248 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.596588 31266 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:22.570789 31144 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.577189 29004 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:22.512691 31139 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.604025 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.605788 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.577596 29125 leader_election.cc:403] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:22.566210 31079 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.585572 29080 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:22.576767 29710 rpcz_store.cc:276] 0709 13:11:21.493764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.531012 (+ 37248us) service_pool.cc:224] Handling call
0709 13:11:22.223697 (+692685us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:22.538627 (+314930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40598272}
W20260709 13:11:22.587253 31269 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:11:22.588740 30958 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:22.590664 31270 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:11:22.591516 31211 consensus_meta.cc:287] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.629s	user 0.003s	sys 0.004s
I20260709 13:11:22.604766 30984 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:22.595145 31255 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:22.581641 31001 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.001s
I20260709 13:11:22.597852 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.613058 31144 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
W20260709 13:11:22.613937 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.639979 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.640520 31224 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.640797 29710 raft_consensus.cc:2393] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:22.642980 31108 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.586624 31025 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:22.620385 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.643981 29715 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:22.620476 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.624711 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.649667 29717 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:22.593950 31263 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.652542 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.627097 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:22.627287 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.632329 31027 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.633586 29712 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:22.615334 31079 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:22.636401 29006 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:22.606627 29004 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:22.638918 29082 raft_consensus.cc:2368] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4.
I20260709 13:11:22.624951 31251 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.624922 31118 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.617537 31248 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:11:22.636765 31211 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.758163 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.758832 29006 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:22.759032 29079 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:22.758294 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.759543 31211 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.759913 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.760726 29006 raft_consensus.cc:2368] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:11:22.756402 30716 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.760916 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.761699 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.656247 29713 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:22.762281 29004 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:22.764197 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.765154 29004 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:22.731604 29010 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:22.764652 29006 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:22.754545 31160 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.766935 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:22.761605 30716 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.763195 31118 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.768522 29715 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:22.767062 30675 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.769488 31249 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.720067 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.766052 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.760747 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.636147 31176 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.644820 31001 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:22.706359 29711 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:22.842250 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.800433 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1926) took 1443 ms. Trace:
I20260709 13:11:22.658450 31190 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.854224 29154 rpcz_store.cc:276] 0709 13:11:21.357469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.357597 (+   128us) service_pool.cc:224] Handling call
0709 13:11:22.800262 (+1442665us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b54001cfcd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:22.800372 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70978432}
I20260709 13:11:22.798805 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.801815 29007 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:11:22.770155 30958 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:22.879873 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:22.886650 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.873788 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 232) took 1783 ms. Trace:
I20260709 13:11:22.856993 29717 raft_consensus.cc:1813] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:22.798252 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.801205 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1544) took 1731 ms. Trace:
W20260709 13:11:22.824959 31273 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:11:22.851043 31187 consensus_meta.cc:287] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.990s	user 0.011s	sys 0.001s
I20260709 13:11:22.887470 29008 rpcz_store.cc:276] 0709 13:11:21.090951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.567979 (+477028us) service_pool.cc:224] Handling call
0709 13:11:22.287344 (+719365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:22.360815 (+ 73471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:22.873534 (+512719us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b5400025650. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:22.873714 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653487360}
I20260709 13:11:22.953696 29151 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:22.849668 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.831701 31205 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.730939 31260 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:22.727149 31212 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.983529 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:22.822106 31225 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:22.864440 31192 consensus_meta.cc:287] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.833s	user 0.012s	sys 0.002s
I20260709 13:11:22.887852 29717 raft_consensus.cc:2410] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:11:22.860834 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:22.987917 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.890086 29005 rpcz_store.cc:276] 0709 13:11:21.069576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.088596 (+ 19020us) service_pool.cc:224] Handling call
0709 13:11:22.800967 (+1712371us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400184cd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:22.801134 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248238336}
I20260709 13:11:22.888700 29006 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:22.953998 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:22.845856 31168 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:22.954617 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:22.849992 31198 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.906260 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:22.993743 29154 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:22.824374 31139 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.882412 29710 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:22.658387 31025 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:22.866731 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:22.998531 29081 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:22.772382 30716 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.798300 31118 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.995540 31260 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:23.000411 31192 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:22.864763 30990 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:22.999989 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.000779 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.044449 29005 raft_consensus.cc:2368] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:23.044992 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1923) took 1070 ms. Trace:
I20260709 13:11:23.045372 29005 rpcz_store.cc:276] 0709 13:11:21.974425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.000562 (+1026137us) service_pool.cc:224] Handling call
0709 13:11:23.044907 (+ 44345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.046510 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.047328 29005 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:23.001477 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:23.002094 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 48176.9us
I20260709 13:11:23.048491 29006 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:23.048940 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1912) took 1082 ms. Trace:
I20260709 13:11:22.988224 31224 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.049285 29006 rpcz_store.cc:276] 0709 13:11:21.966429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.001290 (+1034861us) service_pool.cc:224] Handling call
0709 13:11:23.048889 (+ 47599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:22.885717 31263 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:11:23.056968 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:23.058468 30990 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.011536 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1921) took 1037 ms. Trace:
I20260709 13:11:22.765722 31251 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.059906 29004 rpcz_store.cc:276] 0709 13:11:21.973945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.905896 (+931951us) service_pool.cc:224] Handling call
0709 13:11:23.011475 (+105579us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.060488 31224 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.060820 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.060997 30990 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.061298 31251 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:23.061430 29004 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.060330 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:23.062444 31224 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.015354 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:23.064375 29081 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:23.064795 29003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1540) took 1996 ms (client timeout 2151 ms). Trace:
I20260709 13:11:23.065186 31025 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.993506 31187 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.067286 31187 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:22.776443 31171 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:22.999020 31225 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:23.068414 29002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33750 (request call id 233) took 1951 ms (client timeout 1790 ms). Trace:
W20260709 13:11:23.068795 29002 rpcz_store.cc:269] 0709 13:11:21.117207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.210174 (+ 92967us) service_pool.cc:224] Handling call
0709 13:11:23.068255 (+1858081us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b5400311ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.068348 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75308416}
I20260709 13:11:23.069810 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.069906 31225 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.070603 31225 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:23.070839 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.071645 29002 raft_consensus.cc:2410] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:11:23.071233 31225 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.028738 31260 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.032673 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.034754 31001 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.036324 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:23.008644 31190 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:23.056643 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.004235 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:23.057731 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.121161 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.126701 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.119957 29081 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:23.127450 29006 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:23.075520 31197 consensus_meta.cc:287] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.889s	user 0.005s	sys 0.006s
I20260709 13:11:23.075946 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:23.121932 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 85715.5us
I20260709 13:11:23.129282 29079 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.077343 31173 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:23.077616 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1925) took 1093 ms. Trace:
I20260709 13:11:23.078794 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:23.127948 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:23.134115 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 55395us
I20260709 13:11:23.134538 29081 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:23.134613 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1916) took 1162 ms. Trace:
I20260709 13:11:23.135030 29006 rpcz_store.cc:276] 0709 13:11:21.972161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.126462 (+1154301us) service_pool.cc:224] Handling call
0709 13:11:23.134545 (+  8083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.135439 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:23.065186 29003 rpcz_store.cc:269] 0709 13:11:21.068488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.079278 (+ 10790us) service_pool.cc:224] Handling call
0709 13:11:23.064651 (+1985373us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b54001851d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.064743 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49088256}
I20260709 13:11:23.135823 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.135941 29081 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.136359 29006 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.024305 31266 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:23.037674 30780 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 13:11:23.133695 29010 rpcz_store.cc:276] 0709 13:11:21.983830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.370925 (+387095us) service_pool.cc:224] Handling call
0709 13:11:23.077455 (+706530us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b54001d6fd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.077553 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805563264}
I20260709 13:11:23.125764 29083 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
W20260709 13:11:23.139710 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.140970 29083 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:23.142303 29079 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:23.079895 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1918) took 1099 ms. Trace:
I20260709 13:11:23.143313 29002 rpcz_store.cc:276] 0709 13:11:21.973033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.070615 (+1097582us) service_pool.cc:224] Handling call
0709 13:11:23.072005 (+  1390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.125489 29008 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:23.144538 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1909) took 1178 ms. Trace:
I20260709 13:11:23.144960 29008 rpcz_store.cc:276] 0709 13:11:21.965748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.056364 (+1090616us) service_pool.cc:224] Handling call
0709 13:11:23.144447 (+ 88083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.137312 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:23.137364 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 88110.9us
I20260709 13:11:23.137948 29004 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:23.126726 29005 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:23.146472 29081 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:23.146781 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1928) took 1084 ms. Trace:
I20260709 13:11:23.146178 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:23.442564 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 229) took 2385 ms. Trace:
W20260709 13:11:23.349901 29712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49414 (request call id 368) took 1799 ms (client timeout 1898 ms). Trace:
I20260709 13:11:23.308259 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1938) took 1700 ms. Trace:
I20260709 13:11:23.147085 29005 rpcz_store.cc:276] 0709 13:11:22.062468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.049981 (+987513us) service_pool.cc:224] Handling call
0709 13:11:23.146730 (+ 96749us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.398494 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1490) took 1294 ms. Trace:
I20260709 13:11:23.444954 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.314332 29153 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:23.445885 29082 rpcz_store.cc:276] 0709 13:11:22.103923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.367154 (+263231us) service_pool.cc:224] Handling call
0709 13:11:23.393765 (+1026611us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540027b5d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.398387 (+  4622us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":622249216}
I20260709 13:11:23.443943 29009 rpcz_store.cc:276] 0709 13:11:21.057166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.057360 (+   194us) service_pool.cc:224] Handling call
0709 13:11:21.111515 (+ 54155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:21.379379 (+267864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:21.379426 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:21.395452 (+ 16026us) log.cc:844] Serialized 26 byte log entry
0709 13:11:21.489186 (+ 93734us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:21.489240 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.442051 (+1952811us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54003f20d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:23.442117 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:23.442191 (+    74us) raft_consensus.cc:1685] finished
0709 13:11:23.442212 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.442491 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":8226,"spinlock_wait_cycles":596542720,"thread_start_us":39759,"threads_started":1,"wal-append.queue_time_us":150085,"wal-append.run_cpu_time_us":8281,"wal-append.run_wall_time_us":1573180}
I20260709 13:11:23.445037 29152 rpcz_store.cc:276] 0709 13:11:21.607756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.669638 (+ 61882us) service_pool.cc:224] Handling call
0709 13:11:23.307990 (+1638352us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54002a30d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.308181 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402470784}
I20260709 13:11:23.447319 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.348105 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1538) took 2281 ms. Trace:
W20260709 13:11:23.450152 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 304014us
I20260709 13:11:23.122228 31187 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:23.450875 29050 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:23.239476 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:23.444088 29712 rpcz_store.cc:269] 0709 13:11:21.550307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.550435 (+   128us) service_pool.cc:224] Handling call
0709 13:11:22.322620 (+772185us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:23.349723 (+1027103us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b540048f154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.349847 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642840704}
I20260709 13:11:23.238986 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.454276 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:23.449052 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.254045 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.450346 29001 rpcz_store.cc:276] 0709 13:11:21.066912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.297871 (+230959us) service_pool.cc:224] Handling call
0709 13:11:21.303957 (+  6086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:21.966314 (+662357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.347835 (+1381521us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b540035e3d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:23.348012 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1518512896}
I20260709 13:11:23.128345 31190 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.355005 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.458290 29157 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 3: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:23.218436 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.458951 29003 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:23.459470 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1937) took 1329 ms. Trace:
I20260709 13:11:23.459810 29003 rpcz_store.cc:276] 0709 13:11:22.130210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.136349 (+1006139us) service_pool.cc:224] Handling call
0709 13:11:23.459418 (+323069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.457078 29154 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
W20260709 13:11:23.460088 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.269498 29006 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:23.463559 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1911) took 1497 ms. Trace:
I20260709 13:11:23.463804 29006 rpcz_store.cc:276] 0709 13:11:21.966209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.135622 (+1169413us) service_pool.cc:224] Handling call
0709 13:11:23.463515 (+327893us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.464982 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 236) took 2325 ms. Trace:
I20260709 13:11:23.465307 29008 rpcz_store.cc:276] 0709 13:11:21.139752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.197097 (+2057345us) service_pool.cc:224] Handling call
0709 13:11:23.464660 (+267563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.464781 (+   121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.464797 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:23.464824 (+    27us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:23.464848 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.464879 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.464944 (+    65us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.146974 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:23.456620 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.468815 29004 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:23.468581 29051 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:23.471287 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 324390us
I20260709 13:11:23.472005 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.285902 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:23.326037 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:23.472734 28975 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:23.470453 30780 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:23.482137 30780 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.482522 30780 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:23.482951 29077 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1068) took 1706 ms (client timeout 2175 ms). Trace:
W20260709 13:11:23.483482 29077 rpcz_store.cc:269] 0709 13:11:21.776253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.221570 (+445317us) service_pool.cc:224] Handling call
0709 13:11:23.482761 (+1261191us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540025a654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.482884 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427942272}
I20260709 13:11:23.468454 30950 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:23.474606 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.484453 30950 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.484918 30950 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.485209 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.485893 29154 raft_consensus.cc:2368] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:11:23.486601 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.487069 29154 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.487864 31266 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.449481 31243 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:23.488967 31243 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:23.489334 29156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 1932) took 2130 ms (client timeout 2147 ms). Trace:
I20260709 13:11:23.476521 29050 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:23.489858 29156 rpcz_store.cc:269] 0709 13:11:21.358908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.597155 (+238247us) service_pool.cc:224] Handling call
0709 13:11:22.307557 (+710402us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:23.489168 (+1181611us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b540013d154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.489282 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358039680}
I20260709 13:11:23.490945 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.494050 31204 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.477186 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.477742 31192 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:23.494786 31192 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:23.496187 29155 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:23.478783 31205 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.497073 29003 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:23.480343 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.127933 31260 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.491927 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.488560 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 225) took 1845 ms. Trace:
W20260709 13:11:23.475901 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.500451 29151 rpcz_store.cc:276] 0709 13:11:21.642738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.515050 (+872312us) service_pool.cc:224] Handling call
0709 13:11:22.541347 (+ 26297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:22.985156 (+443809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.488317 (+503161us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400141ed4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:23.488485 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947714944}
I20260709 13:11:23.354895 31176 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.471287 31215 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.495687 31192 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.505746 31144 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:23.506148 31144 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:23.478238 31212 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.506947 31144 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.507110 31212 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.496699 31205 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.479303 31221 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.497782 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.498466 29154 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:23.470620 31198 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:23.512183 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.354179 31225 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.148991 31001 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:23.470788 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1927) took 1468 ms. Trace:
W20260709 13:11:23.508539 29080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 249) took 2495 ms (client timeout 3065 ms). Trace:
I20260709 13:11:23.512447 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.271405 31061 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.510129 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 234) took 2389 ms. Trace:
I20260709 13:11:23.533378 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.137385 31173 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:23.510439 29085 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:11:23.489351 31243 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.514077 29156 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:23.577142 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.577849 29156 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:23.510730 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1169) took 1604 ms. Trace:
I20260709 13:11:23.513044 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.137367 31197 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.453326 31187 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.325469 31092 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:23.511327 31205 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.197731 31241 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.349643 31214 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:23.469930 31190 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.560371 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.587443 29001 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:23.533439 29007 rpcz_store.cc:276] 0709 13:11:21.120659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.584010 (+463351us) service_pool.cc:224] Handling call
0709 13:11:22.801467 (+1217457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:22.889794 (+ 88327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.509959 (+620165us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400184cd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.510077 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2798936448}
I20260709 13:11:23.562604 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.589268 29009 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:23.562827 29151 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:23.199257 31171 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:23.563678 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:23.551494 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:23.517429 29080 rpcz_store.cc:269] 0709 13:11:21.012877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.014184 (+  1307us) service_pool.cc:224] Handling call
0709 13:11:23.508323 (+2494139us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54000c8ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.508461 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3488366848}
I20260709 13:11:23.519562 29002 rpcz_store.cc:276] 0709 13:11:22.002777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.144175 (+1141398us) service_pool.cc:224] Handling call
0709 13:11:23.470708 (+326533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.508014 31144 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.565862 31001 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:23.511605 31221 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.516438 29155 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.353485 31095 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.578261 29159 rpcz_store.cc:276] 0709 13:11:21.907588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.867649 (+960061us) service_pool.cc:224] Handling call
0709 13:11:22.875867 (+  8218us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:22.876185 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:22.876221 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:22.896939 (+ 20718us) log.cc:844] Serialized 26 byte log entry
0709 13:11:23.140317 (+243378us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:23.140351 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.140412 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:23.510533 (+370121us) raft_consensus.cc:1685] finished
0709 13:11:23.510552 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.510677 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":136242,"threads_started":1,"wal-append.queue_time_us":511873}
I20260709 13:11:23.585302 31187 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.611856 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.612694 29001 raft_consensus.cc:2368] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:11:23.512108 29005 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:23.509446 31212 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.285049 31079 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.591312 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.565737 31061 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.565773 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.623014 29010 raft_consensus.cc:2368] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:11:23.595192 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.595171 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:23.600232 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:23.597169 31177 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.111s	user 0.009s	sys 0.013s
I20260709 13:11:23.578956 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.599076 31001 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.596513 31205 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:23.588613 30780 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.592620 31198 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:11:23.592031 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 90493.1us
I20260709 13:11:23.624182 29154 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 3.
I20260709 13:11:23.611856 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.660434 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:23.660575 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.661409 29154 raft_consensus.cc:2410] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:23.660459 29151 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:23.624161 29002 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.661723 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:23.625152 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.662678 29001 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:23.663255 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:23.663532 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.664168 29001 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:23.664945 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.623831 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.665674 29159 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:23.592200 31241 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.596308 30950 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.592139 31092 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.633692 31195 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:23.631986 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.679764 29009 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.634872 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.681221 29007 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.598650 31171 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:23.636161 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.678135 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.689239 29010 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:23.689942 29151 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:23.626543 31177 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.209s	user 0.009s	sys 0.014s
I20260709 13:11:23.622331 31287 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:23.649688 30780 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.657527 31061 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:23.667001 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:23.696839 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 234278us
I20260709 13:11:23.697324 29156 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:23.621254 31204 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:23.626745 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.698807 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.699059 29002 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:23.670393 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:23.626598 31095 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:23.669626 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.704921 29680 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:23.706009 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.630460 28976 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:23.676739 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.677392 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:23.682015 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 205547us
I20260709 13:11:23.659032 31214 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.692379 29078 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:23.659355 31198 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:23.671007 31225 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:23.667990 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.701612 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:23.670128 31260 raft_consensus.cc:2764] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:11:23.706779 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 76386.2us
I20260709 13:11:23.730054 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:23.710888 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:23.697405 31287 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.733091 31287 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.736989 29048 leader_election.cc:400] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:23.737419 29048 leader_election.cc:403] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:11:23.738507 31260 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:23.738934 31079 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.706919 29159 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.741644 31287 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.688750 31211 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.744343 31211 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.744959 31221 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.669847 31176 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:11:23.746001 31197 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.716430 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.746492 31287 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.714419 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.719870 29154 raft_consensus.cc:2410] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:23.696364 31215 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:23.747880 31215 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:23.748291 31215 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.748417 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.749140 29154 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.749775 31156 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:23.720273 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.752245 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.720868 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.728228 30780 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:23.753372 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.754649 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.754949 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1167) took 1857 ms. Trace:
W20260709 13:11:23.756397 29150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1165) took 1881 ms (client timeout 2195 ms). Trace:
W20260709 13:11:23.756877 29150 rpcz_store.cc:269] 0709 13:11:21.895044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.154402 (+259358us) service_pool.cc:224] Handling call
0709 13:11:23.756224 (+1601822us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b5400139f54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.756341 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":797813120}
I20260709 13:11:23.747010 29005 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:23.738935 31260 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:23.753450 30780 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:23.771970 29155 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:23.754941 29002 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:23.787611 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:23.728595 31171 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.721738 31266 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:23.760491 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:23.760584 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.763549 29159 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:23.808508 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:23.755331 29153 rpcz_store.cc:276] 0709 13:11:21.898012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.706191 (+808179us) service_pool.cc:224] Handling call
0709 13:11:23.308546 (+602355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.446398 (+137852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.754768 (+308370us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54002a30d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.754895 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384376064}
I20260709 13:11:23.810171 29153 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:11:23.810676 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1179) took 1837 ms. Trace:
I20260709 13:11:23.810940 29153 rpcz_store.cc:276] 0709 13:11:21.973339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.809744 (+1836405us) service_pool.cc:224] Handling call
0709 13:11:23.809985 (+   241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:23.810503 (+   518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.810637 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.711753 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:23.812618 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 100943us
I20260709 13:11:23.774353 29009 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:23.753031 31241 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.816846 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.817544 29010 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:23.788321 29150 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:23.735303 31205 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:23.820201 29007 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
W20260709 13:11:23.788763 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:23.820847 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 60438.4us
I20260709 13:11:23.709429 31177 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:23.788977 29155 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:23.788718 29154 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:23.714699 31204 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:23.788439 30780 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.717929 31214 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.718158 31198 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.715883 31170 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:23.797276 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:23.858716 29005 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.746647 31197 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:23.719139 31225 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.806521 31092 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.717810 31200 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:23.813776 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.819730 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:23.821300 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.820156 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.824383 31195 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:23.824491 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.782519 31260 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.789646 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1472) took 2210 ms. Trace:
I20260709 13:11:23.714511 31139 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:11:23.858304 31170 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.793147 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:23.833992 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.793777 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:23.931702 29006 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.932353 29004 raft_consensus.cc:2410] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:23.807330 31266 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.820536 31205 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.825524 31156 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:23.826207 31004 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:11:23.860306 30950 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.862119 31214 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:23.864785 31173 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.936920 31170 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:23.880579 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 143771us
I20260709 13:11:23.958688 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:23.895357 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.941681 31205 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:23.964568 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1170) took 2056 ms. Trace:
I20260709 13:11:23.964994 29156 rpcz_store.cc:276] 0709 13:11:21.907924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.697991 (+1790067us) service_pool.cc:224] Handling call
0709 13:11:23.713848 (+ 15857us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:23.729894 (+ 16046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.729927 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:23.771501 (+ 41574us) log.cc:844] Serialized 52 byte log entry
0709 13:11:23.854661 (+ 83160us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:23.854720 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.854818 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:23.964321 (+109503us) raft_consensus.cc:1685] finished
0709 13:11:23.964342 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.964496 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":709,"raft.queue_time_us":3058,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6710,"thread_start_us":54594,"threads_started":1,"wal-append.queue_time_us":141784}
I20260709 13:11:23.965703 31215 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.966660 29001 raft_consensus.cc:2393] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 3: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:23.967414 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 233) took 1374 ms. Trace:
I20260709 13:11:23.896222 31225 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:23.935750 29004 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:23.980942 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1913) took 2014 ms. Trace:
W20260709 13:11:23.939325 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.898885 31177 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.894817 31144 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:23.896754 31263 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:23.949126 29005 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:23.864825 31190 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:23.874500 31187 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:23.954174 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 244) took 2306 ms. Trace:
I20260709 13:11:23.939021 31266 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:23.966046 31173 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:23.975234 31102 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:23.923884 29084 rpcz_store.cc:276] 0709 13:11:21.582428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.582543 (+   115us) service_pool.cc:224] Handling call
0709 13:11:23.789486 (+2206943us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54001ad1d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:23.789593 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502547840}
I20260709 13:11:23.826350 29154 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:23.999361 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 229) took 1967 ms. Trace:
I20260709 13:11:23.999835 29154 rpcz_store.cc:276] 0709 13:11:22.031662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.825511 (+1793849us) service_pool.cc:224] Handling call
0709 13:11:23.826189 (+   678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.999026 (+172837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.999288 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738048}
I20260709 13:11:24.001415 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:24.002357 29154 raft_consensus.cc:2393] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 3: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:24.003719 29154 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:23.941869 31200 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:23.970387 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:23.968391 29001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34922
I20260709 13:11:24.007355 29001 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:24.007689 29001 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:23.982009 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:24.008395 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:24.008205 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 188499us
I20260709 13:11:24.009578 29048 leader_election.cc:400] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:11:24.009941 29048 leader_election.cc:403] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:11:23.981634 29006 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:23.985414 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1922) took 2011 ms. Trace:
I20260709 13:11:24.010491 29005 rpcz_store.cc:276] 0709 13:11:21.974209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.984659 (+2010450us) service_pool.cc:224] Handling call
0709 13:11:23.984948 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.985121 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.985147 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:23.985186 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:23.985227 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.985275 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.985366 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.967751 31215 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:23.989395 29010 rpcz_store.cc:276] 0709 13:11:21.647711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.941593 (+2293882us) service_pool.cc:224] Handling call
0709 13:11:23.953641 (+ 12048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.953829 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.953855 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:23.953895 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:23.953926 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.953977 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.954106 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.981288 29004 rpcz_store.cc:276] 0709 13:11:21.966647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.935100 (+1968453us) service_pool.cc:224] Handling call
0709 13:11:23.935490 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.980695 (+ 45205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.980886 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.015460 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:24.016003 29004 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:23.967798 29156 rpcz_store.cc:276] 0709 13:11:22.593468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.966579 (+1373111us) service_pool.cc:224] Handling call
0709 13:11:23.966921 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.967098 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.967127 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:23.967170 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:23.967208 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.967263 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.967360 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:23.991317 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 148743us
I20260709 13:11:23.956403 31156 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:23.970613 30950 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:23.972375 29008 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:24.004751 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1948) took 1130 ms. Trace:
I20260709 13:11:23.935240 31204 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.014428 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:24.048710 29006 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:24.019368 31217 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:11:24.020339 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.021131 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.022094 29010 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:24.060784 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 246) took 1477 ms. Trace:
I20260709 13:11:24.061254 29010 rpcz_store.cc:276] 0709 13:11:22.583234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.021661 (+1438427us) service_pool.cc:224] Handling call
0709 13:11:24.021935 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.060453 (+ 38518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.060718 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.021844 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.022567 31294 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:24.023034 31092 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.023193 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 1954) took 1141 ms. Trace:
I20260709 13:11:23.973420 31243 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.070186 29084 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:11:24.070890 31243 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.071113 31241 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.989195 31144 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.071452 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.071650 31241 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.072463 31156 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.038460 31184 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.073242 31144 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.035754 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.074677 29077 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:24.036020 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.025003 29001 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.039176 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.075955 29714 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:24.035451 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.077301 29080 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:24.078444 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.079684 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.080443 29080 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:24.081432 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.039294 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:24.039965 29154 rpcz_store.cc:276] 0709 13:11:22.874782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.003163 (+1128381us) service_pool.cc:224] Handling call
0709 13:11:24.003498 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.004462 (+   964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.004688 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.064193 29156 rpcz_store.cc:276] 0709 13:11:22.881467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.016894 (+1135427us) service_pool.cc:224] Handling call
0709 13:11:24.022682 (+  5788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.022848 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.022873 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.022916 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:24.022951 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.023027 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.023138 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.029260 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:24.039463 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:24.083112 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53970.1us
I20260709 13:11:24.071352 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.083850 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:24.084244 29084 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:24.070621 31001 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.030076 29048 leader_election.cc:400] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
W20260709 13:11:24.091049 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 77688.6us
I20260709 13:11:24.092480 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:24.093814 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:24.071796 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.072919 31249 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.075321 29082 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:24.036655 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.107905 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:23.996779 31263 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.108470 29079 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:24.054342 31215 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:24.036379 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.068264 31294 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:24.069092 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 247) took 1485 ms. Trace:
I20260709 13:11:24.118685 29006 rpcz_store.cc:276] 0709 13:11:22.583494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.068379 (+1484885us) service_pool.cc:224] Handling call
0709 13:11:24.068659 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.068813 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.068841 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.068876 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:24.068909 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.068956 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.069040 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:23.987824 31177 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.073594 31184 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.088804 29001 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.083061 29125 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:24.126240 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:24.126745 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43770us
I20260709 13:11:24.127142 29084 raft_consensus.cc:2368] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:24.094656 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:24.129060 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:24.021795 31205 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.099607 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1190) took 1025 ms. Trace:
I20260709 13:11:24.107838 31217 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:24.102272 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.102703 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.034014 31214 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:24.073675 31241 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.037582 31200 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:24.065560 31197 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.053814 31171 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:24.109513 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.038852 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.110085 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.158757 29082 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:24.159920 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.160887 29082 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:24.111155 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 230) took 2077 ms. Trace:
I20260709 13:11:24.111752 29156 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:24.162354 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1193) took 1084 ms. Trace:
I20260709 13:11:24.162921 29156 rpcz_store.cc:276] 0709 13:11:23.077448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.082760 (+1005312us) service_pool.cc:224] Handling call
0709 13:11:24.102888 (+ 20128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.162017 (+ 59129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.162283 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.161849 29155 rpcz_store.cc:276] 0709 13:11:22.034023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.827024 (+1793001us) service_pool.cc:224] Handling call
0709 13:11:23.857855 (+ 30831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.858061 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:23.983000 (+124939us) log.cc:844] Serialized 14 byte log entry
0709 13:11:24.110162 (+127162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.110237 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:24.110377 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.110572 (+   195us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.111020 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":189730,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":5450,"raft.queue_time_us":161,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7986,"thread_start_us":211076,"threads_started":2}
I20260709 13:11:24.117612 29002 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:24.118278 29081 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:24.066370 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1932) took 1967 ms. Trace:
I20260709 13:11:24.090096 31243 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.087525 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:24.091256 31209 raft_consensus.cc:2764] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:11:24.090837 31173 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:24.124181 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1182) took 2137 ms. Trace:
I20260709 13:11:24.127777 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:24.131374 29010 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:24.131901 29157 rpcz_store.cc:276] 0709 13:11:23.074482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.068039 (+993557us) service_pool.cc:224] Handling call
0709 13:11:24.098799 (+ 30760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.099190 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.099231 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.099286 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:24.099331 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.099404 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.099537 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1031,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":7951}
I20260709 13:11:24.100976 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.105718 30950 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:24.133841 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:34922 (request call id 1914) took 2159 ms. Trace:
I20260709 13:11:24.134398 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 241) took 2919 ms. Trace:
I20260709 13:11:24.135535 31217 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:24.137394 29715 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:24.149343 31184 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:24.151937 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.074693 31092 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.117440 29711 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:24.123301 31294 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:24.226735 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1173) took 2317 ms. Trace:
W20260709 13:11:24.197031 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.234283 29077 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:24.290520 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.289558 29159 rpcz_store.cc:276] 0709 13:11:21.908880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.809425 (+1900545us) service_pool.cc:224] Handling call
0709 13:11:23.817604 (+  8179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:23.817934 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.817964 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:23.827035 (+  9071us) log.cc:844] Serialized 26 byte log entry
0709 13:11:23.984899 (+157864us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:23.984954 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.985057 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:24.226467 (+241410us) raft_consensus.cc:1685] finished
0709 13:11:24.226498 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.226660 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":555,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5840,"thread_start_us":87423,"threads_started":1,"wal-append.queue_time_us":261424}
I20260709 13:11:24.290977 29004 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:24.291188 29077 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:24.218367 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 179132us
I20260709 13:11:24.224987 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1547) took 2815 ms. Trace:
I20260709 13:11:24.262488 29001 rpcz_store.cc:276] 0709 13:11:21.971438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.968283 (+1996845us) service_pool.cc:224] Handling call
0709 13:11:24.126040 (+157757us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":442}
I20260709 13:11:24.241010 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:24.293272 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 165595us
I20260709 13:11:24.293853 29152 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:24.205852 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.260862 29009 rpcz_store.cc:276] 0709 13:11:21.214842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.898778 (+2683936us) service_pool.cc:224] Handling call
0709 13:11:23.899205 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.899473 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:24.033492 (+134019us) log.cc:844] Serialized 14 byte log entry
0709 13:11:24.133953 (+100461us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.134019 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:24.134084 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.134170 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.134328 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":282967,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":96,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17638,"thread_start_us":193677,"threads_started":2}
I20260709 13:11:24.292622 29007 rpcz_store.cc:276] 0709 13:11:21.409571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.922080 (+2512509us) service_pool.cc:224] Handling call
0709 13:11:23.922430 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:23.922790 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.922814 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:23.923289 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:11:24.044774 (+121485us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:24.044813 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.044878 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:24.224711 (+179833us) raft_consensus.cc:1685] finished
0709 13:11:24.224736 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.224915 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8394,"thread_start_us":93276,"threads_started":1,"wal-append.queue_time_us":198641}
I20260709 13:11:24.261186 29005 rpcz_store.cc:276] 0709 13:11:22.099104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.032642 (+1933538us) service_pool.cc:224] Handling call
0709 13:11:24.065897 (+ 33255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.066066 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.066093 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.066131 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:24.066159 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.066225 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.066319 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.295111 29009 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:24.239614 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.274132 29084 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:24.295703 29083 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:24.272878 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:24.273258 29080 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:24.274984 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:24.279810 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 192389us
I20260709 13:11:24.296893 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.278008 29153 rpcz_store.cc:276] 0709 13:11:21.986411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.811509 (+1825098us) service_pool.cc:224] Handling call
0709 13:11:23.811712 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.811859 (+   147us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:23.897703 (+ 85844us) log.cc:844] Serialized 14 byte log entry
0709 13:11:24.123249 (+225546us) log.cc:844] Serialized 14 byte log entry
0709 13:11:24.123681 (+   432us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.123741 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:24.123804 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.123949 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.124109 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9347,"prepare.queue_time_us":201784,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":10628,"raft.queue_time_us":9541,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":5043,"spinlock_wait_cycles":37226496,"thread_start_us":286826,"threads_started":2,"wal-append.queue_time_us":323616}
I20260709 13:11:24.300078 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:24.165205 29155 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:24.294692 29082 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:24.301046 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:24.140514 31061 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:24.294202 29125 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:24.309924 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:24.310525 29157 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.311188 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:24.282092 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:24.311944 29010 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:11:24.312326 29010 pending_rounds.cc:85] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 2
I20260709 13:11:24.312554 29010 pending_rounds.cc:107] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:11:24.312785 29010 raft_consensus.cc:2889] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:24.313426 29050 leader_election.cc:400] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:11:24.315199 29050 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:24.301580 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 236) took 1083 ms. Trace:
I20260709 13:11:24.125002 31215 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.319247 31215 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.139750 31287 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:24.320767 29155 rpcz_store.cc:276] 0709 13:11:23.218194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.164449 (+946255us) service_pool.cc:224] Handling call
0709 13:11:24.164891 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:24.301339 (+136448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.301530 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.306562 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:24.322767 29159 raft_consensus.cc:2368] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:24.322651 31221 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:24.324632 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 238) took 1039 ms. Trace:
I20260709 13:11:24.305135 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:24.164367 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:24.223183 31295 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:11:24.325704 31220 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:24.326327 29156 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:24.326671 29156 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:24.325171 31221 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:11:24.325174 29159 rpcz_store.cc:276] 0709 13:11:23.284985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.323697 (+1038712us) service_pool.cc:224] Handling call
0709 13:11:24.324072 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.324263 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.324296 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.324349 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:24.324399 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.324461 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.324572 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.328418 31195 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:24.328980 31195 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:24.329387 31195 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:24.330628 29159 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:24.334682 29125 leader_election.cc:400] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 2 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 2. Current term is 3.
I20260709 13:11:24.335068 29005 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:11:24.335507 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:24.231336 31302 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:24.228446 31177 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.321484 31173 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:24.296944 31241 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.374117 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 238) took 3194 ms. Trace:
I20260709 13:11:24.374508 29008 rpcz_store.cc:276] 0709 13:11:21.179574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.855563 (+2675989us) service_pool.cc:224] Handling call
0709 13:11:23.972213 (+116650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.033866 (+ 61653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.373891 (+340025us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54003aa050. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:24.374061 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267431296}
I20260709 13:11:24.221256 31139 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.302379 29153 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:24.336181 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:24.332906 31095 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:24.232228 31187 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.229534 31205 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.287189 31217 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:24.400307 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.332628 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:24.236449 30921 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:24.327510 29156 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.401094 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 224) took 2762 ms. Trace:
I20260709 13:11:24.352092 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.193931 31171 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:24.410866 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 371431us
I20260709 13:11:24.307668 31243 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.225457 31001 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.321626 29081 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:24.228236 31263 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.311803 31184 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:24.332924 31108 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:24.332661 30958 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:24.158286 31214 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.310132 31061 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:24.231935 30780 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.336870 31144 raft_consensus.cc:2764] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:11:24.189215 31198 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.338712 31287 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:24.348140 31247 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:24.181514 31197 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.401468 29158 rpcz_store.cc:276] 0709 13:11:21.639103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.383401 (+744298us) service_pool.cc:224] Handling call
0709 13:11:22.383674 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:22.403220 (+ 19546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:22.403239 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:22.403563 (+   324us) log.cc:844] Serialized 26 byte log entry
0709 13:11:24.040678 (+1637115us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b1800463060. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:11:24.154215 (+113537us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:24.154252 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.154319 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:24.400710 (+246391us) raft_consensus.cc:1685] finished
0709 13:11:24.400737 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.401010 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21331,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":75920,"spinlock_wait_cycles":8087808,"thread_start_us":113342,"threads_started":1,"wal-append.queue_time_us":1870255}
I20260709 13:11:24.401641 29156 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:24.347921 31004 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:11:24.408362 31177 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:24.425323 31144 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:24.425586 31177 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:24.425779 31144 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:24.426028 31177 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.426185 31144 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:24.412539 30921 raft_consensus.cc:697] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:24.428790 30921 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.429322 30780 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.431003 31171 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.353608 31310 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:11:24.408519 31217 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.375283 31215 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:24.154117 31291 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:24.432351 31215 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:24.432694 31291 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:24.432809 31215 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:24.190750 31190 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.433611 31217 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.433285 31287 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.435474 31177 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.435864 31214 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:24.384333 31241 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.436354 31004 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:24.436764 31004 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:24.437248 29680 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:24.437714 31241 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.437547 31004 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.438730 31004 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.439079 31177 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.414367 31184 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:24.396631 31211 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.390077 29709 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:24.439725 31184 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.440570 31170 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.442862 29151 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:24.442556 31266 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.382290 31139 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.423668 31247 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:24.444283 31247 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.444828 31139 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.433629 31291 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.446472 31184 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.182062 31200 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.432348 31225 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.449048 29156 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:11:24.449734 31156 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:24.450237 31156 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:24.416426 31041 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:11:24.450734 31156 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.451396 31310 raft_consensus.cc:3055] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:24.451776 31310 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:24.452541 31310 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.453688 31310 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:24.443949 31215 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.425369 31197 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.456867 29708 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:24.457552 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.457847 31156 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.459064 31215 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.459883 31156 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:24.449044 31184 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.424444 31092 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.439134 31198 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.464947 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.466012 29715 raft_consensus.cc:2410] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:11:24.466207 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.466436 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.468274 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.468919 29712 raft_consensus.cc:2368] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:24.470147 31210 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.470794 31210 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.419456 31061 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:24.475911 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.477029 29714 raft_consensus.cc:2410] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:11:24.466101 31243 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.468823 31095 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:24.480444 31095 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.480501 31210 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.484844 31294 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.485656 31195 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.450130 31225 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.491521 31266 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:24.494596 31302 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:24.498503 29708 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:24.380348 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1549) took 1017 ms. Trace:
I20260709 13:11:24.340993 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:24.415125 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1489) took 2311 ms. Trace:
I20260709 13:11:24.604411 29002 rpcz_store.cc:276] 0709 13:11:23.362620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.784957 (+422337us) service_pool.cc:224] Handling call
0709 13:11:24.380000 (+595043us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54001d74d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:24.380277 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497830784}
I20260709 13:11:24.303983 29001 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:24.605186 29078 rpcz_store.cc:276] 0709 13:11:22.103607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.515561 (+411954us) service_pool.cc:224] Handling call
0709 13:11:22.515806 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.709818 (+1194012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.414908 (+705090us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540025ab54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:24.415066 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2697958272}
I20260709 13:11:24.606640 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1570) took 1014 ms. Trace:
I20260709 13:11:24.380391 29003 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:24.607285 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.607784 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:11:24.608610 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 267603us
I20260709 13:11:24.402160 29084 raft_consensus.cc:2410] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:11:24.609081 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1936) took 2486 ms. Trace:
I20260709 13:11:24.550652 29079 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:24.609980 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.610736 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.611032 29717 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:24.611227 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:24.612269 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.612543 29078 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 3: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:24.551132 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1562) took 1090 ms. Trace:
I20260709 13:11:24.352488 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:24.613525 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.614854 31209 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:11:24.401203 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 65764.6us
I20260709 13:11:24.410885 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1486) took 2308 ms. Trace:
I20260709 13:11:24.607055 29001 rpcz_store.cc:276] 0709 13:11:23.591832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.303576 (+711744us) service_pool.cc:224] Handling call
0709 13:11:24.303837 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.606062 (+302225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.606450 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:24.574415 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.616460 29085 rpcz_store.cc:276] 0709 13:11:22.102559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.511234 (+408675us) service_pool.cc:224] Handling call
0709 13:11:23.510202 (+998968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:23.566538 (+ 56336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.410708 (+844170us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54000c8ad0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:24.410832 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296781696}
I20260709 13:11:24.495281 31247 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.618273 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.618135 31025 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:24.619652 29085 raft_consensus.cc:2410] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:24.416006 29157 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:24.620210 31317 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:24.620862 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:24.622314 29159 raft_consensus.cc:2368] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
W20260709 13:11:24.555052 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.627372 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:24.629334 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:24.630272 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:24.631531 29153 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:24.632485 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:24.634934 30921 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:24.636381 30921 raft_consensus.cc:3055] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:24.636929 30921 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:24.637511 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.510715 31195 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:24.551949 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.638808 30921 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.613552 29006 rpcz_store.cc:276] 0709 13:11:23.460674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.119916 (+659242us) service_pool.cc:224] Handling call
0709 13:11:24.120424 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:24.122079 (+  1655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.122113 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:24.122587 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:11:24.308747 (+186160us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:24.308802 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.308888 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:24.550410 (+241522us) raft_consensus.cc:1685] finished
0709 13:11:24.550457 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.550830 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1231,"raft.queue_time_us":1404,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":7077,"thread_start_us":125687,"threads_started":1,"wal-append.queue_time_us":268429}
I20260709 13:11:24.643585 31291 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:24.577759 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:24.644992 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.645398 31287 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:11:24.645776 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 71724.9us
I20260709 13:11:24.646005 29085 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:24.644253 31291 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:24.578150 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.646716 31291 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:24.577803 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:24.648546 29714 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:24.650352 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1197) took 1082 ms. Trace:
I20260709 13:11:24.650887 29151 rpcz_store.cc:276] 0709 13:11:23.567296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.693409 (+126113us) service_pool.cc:224] Handling call
0709 13:11:24.649779 (+956370us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54002b66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:24.650081 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598314368}
I20260709 13:11:24.588922 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.653100 29077 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:24.589059 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:24.655030 31312 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:24.609592 29003 rpcz_store.cc:276] 0709 13:11:22.122402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.021130 (+1898728us) service_pool.cc:224] Handling call
0709 13:11:24.380231 (+359101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.607739 (+227508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.608694 (+   955us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54001d74d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:24.608849 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825091328}
I20260709 13:11:24.656961 31139 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:24.657433 31139 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:24.590349 31310 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:24.591267 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.659772 29080 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.662670 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:24.663607 29001 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.593108 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:24.664616 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 71973.3us
I20260709 13:11:24.665503 29122 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:24.666970 31249 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:24.667552 31249 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:24.668099 31249 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:24.668993 29003 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:24.603839 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1171) took 2695 ms. Trace:
I20260709 13:11:24.669586 29150 rpcz_store.cc:276] 0709 13:11:21.908308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.757704 (+1849396us) service_pool.cc:224] Handling call
0709 13:11:23.787012 (+ 29308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.818900 (+ 31888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.603229 (+784329us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b54002a44d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:24.603547 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66656000}
I20260709 13:11:24.597843 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:24.623394 29084 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:24.608755 31027 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:24.672302 31027 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:24.672446 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:24.672834 31027 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:24.673142 31317 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:24.673689 31184 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:24.674292 29081 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:24.673693 31317 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:24.674911 31317 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:24.615027 29711 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:11:24.615476 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 262931us
I20260709 13:11:24.615688 31209 raft_consensus.cc:697] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:24.677855 31209 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.678781 29002 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:24.552943 29076 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:24.676093 29084 raft_consensus.cc:1240] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:24.680761 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:24.682551 29050 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:24.682706 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:24.683578 31184 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:24.442567 31170 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:24.499651 31260 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.564s	user 0.006s	sys 0.003s
I20260709 13:11:24.683657 31225 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:24.684917 31260 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.655026 29077 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:24.686434 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:24.687350 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.687517 31225 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:24.625352 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:24.625329 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:24.626003 29155 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:24.623343 31266 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:11:24.626257 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:24.504856 31294 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.611754 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.658564 31139 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.638762 29082 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:24.668531 31310 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.670661 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:24.683967 31287 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:24.621469 31247 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:24.687912 31260 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.707762 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:24.646946 29051 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:24.647168 29085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "243d4ae4a4ae48b2979e32874472d256"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:24.613687 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:24.687933 31225 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:24.694159 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.709949 31225 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:24.688702 29122 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:24.711656 29157 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:11:24.712918 31260 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:24.713294 29158 raft_consensus.cc:2393] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:24.713523 31260 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:24.713964 31310 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:24.708248 29079 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.714697 31260 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.716210 31210 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:24.707928 31139 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:24.714711 31266 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:24.669085 30921 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:24.701021 29683 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:24.705118 29081 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:11:24.723483 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 63474.2us
I20260709 13:11:24.724938 31195 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:24.725896 31302 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:24.728907 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:24.734175 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:24.735411 31247 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:24.735889 31247 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:24.736567 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:24.736843 31247 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.738222 31027 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:24.740109 29085 raft_consensus.cc:606] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Received request to transfer leadership to TS 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:24.740497 29085 raft_consensus.cc:614] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:11:24.768142 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 431807us
W20260709 13:11:24.771390 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 124643us
I20260709 13:11:24.780720 29080 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
W20260709 13:11:24.785113 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 84360.8us
I20260709 13:11:24.787222 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:24.784562 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:24.791927 31210 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:24.792310 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 63722us
I20260709 13:11:24.792613 31210 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.795553 31210 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:24.796856 29001 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:24.797410 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:24.797490 31170 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.791774 31266 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:24.790025 29051 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:24.799614 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 92117.7us
I20260709 13:11:24.779335 31302 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:11:24.785825 31027 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:24.804206 31169 consensus_queue.cc:1059] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:11:24.845954 28975 leader_election.cc:400] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:24.944497 29081 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:24.798349 29680 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:24.844635 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.844581 29158 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:24.829988 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60692.5us
W20260709 13:11:25.083806 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:25.365043 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:25.365631 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:25.253249 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.057683 29079 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:24.786669 31310 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:24.940454 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:24.807405 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71002.5us
W20260709 13:11:25.368695 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.375078 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1513) took 1319 ms. Trace:
I20260709 13:11:25.375450 29079 rpcz_store.cc:276] 0709 13:11:24.050659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.611376 (+560717us) service_pool.cc:224] Handling call
0709 13:11:25.369803 (+758427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:25.376578 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 266) took 1241 ms. Trace:
W20260709 13:11:24.828888 29076 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1473) took 3246 ms (client timeout 3165 ms). Trace:
I20260709 13:11:25.377003 29081 rpcz_store.cc:276] 0709 13:11:24.130638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.831270 (+700632us) service_pool.cc:224] Handling call
0709 13:11:24.916927 (+ 85657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.369439 (+452512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.372197 (+  2758us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.839536 31317 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.836895 31249 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:25.381028 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 595705us
W20260709 13:11:25.378194 29076 rpcz_store.cc:269] 0709 13:11:21.582836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.582971 (+   135us) service_pool.cc:224] Handling call
0709 13:11:24.828638 (+3245667us) spinlock_profiling.cc:243] Waited 2.96 s on lock 0x7b5400402254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:24.828786 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6798314368}
I20260709 13:11:24.844975 31184 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.538s
I20260709 13:11:25.383188 29051 leader_election.cc:400] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:25.383249 29716 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:25.383616 29051 leader_election.cc:403] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:24.909344 31266 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:25.384457 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 270) took 1194 ms. Trace:
I20260709 13:11:25.384868 29076 rpcz_store.cc:276] 0709 13:11:24.190231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:25.383403 (+1193172us) service_pool.cc:224] Handling call
0709 13:11:25.383762 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.384090 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:25.384124 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.384172 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:25.384213 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.384268 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.384382 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:25.384997 31266 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.387341 29081 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:11:25.387713 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 541936us
I20260709 13:11:24.948872 31095 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.388630 28975 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:25.382654 31249 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:24.935446 31144 consensus_meta.cc:287] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.010s
W20260709 13:11:25.233899 31225 consensus_meta.cc:287] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.001s
I20260709 13:11:25.389611 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1192) took 2314 ms. Trace:
W20260709 13:11:25.395138 29085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.018s	sys 0.043s
W20260709 13:11:25.395503 29085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.018s	sys 0.043s
W20260709 13:11:25.395792 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 597519us
W20260709 13:11:25.395503 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 570547us
I20260709 13:11:25.397336 29078 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:25.383895 31184 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:25.396662 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 271) took 1203 ms. Trace:
I20260709 13:11:25.402575 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 262) took 1293 ms. Trace:
I20260709 13:11:25.402410 29081 rpcz_store.cc:276] 0709 13:11:24.190499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:25.386666 (+1196167us) service_pool.cc:224] Handling call
0709 13:11:25.387024 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.387750 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.393423 (+  5673us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.830395 31328 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:25.403146 29078 rpcz_store.cc:276] 0709 13:11:24.109219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:25.393951 (+1284732us) service_pool.cc:224] Handling call
0709 13:11:25.397164 (+  3213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.402321 (+  5157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.402521 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:25.382680 31317 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:25.406047 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:24.842734 31061 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:24.946538 31291 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.394418 29154 rpcz_store.cc:276] 0709 13:11:23.074995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.082706 (+1007711us) service_pool.cc:224] Handling call
0709 13:11:24.083067 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.420975 (+337908us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:24.432244 (+ 11269us) log.cc:844] Serialized 14 byte log entry
0709 13:11:24.716721 (+284477us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.716814 (+    93us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:24.716897 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.389224 (+672327us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b5400398d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:25.389268 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.389424 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11513,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":146,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":146,"spinlock_wait_cycles":776607488,"thread_start_us":283496,"threads_started":1,"wal-append.queue_time_us":362282}
I20260709 13:11:25.408927 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:25.400595 31177 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:25.410799 31310 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:24.844930 31170 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:25.403753 31184 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:25.418817 29683 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:25.426908 29082 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:25.427469 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1087) took 1357 ms. Trace:
I20260709 13:11:25.427798 29082 rpcz_store.cc:276] 0709 13:11:24.069788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:25.393831 (+1324043us) service_pool.cc:224] Handling call
0709 13:11:25.394177 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.427282 (+ 33105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.427430 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:25.403017 31144 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.430537 29079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab75774d33164f0b9c0269be22426768"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:44418
I20260709 13:11:25.432389 31095 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:25.433463 31061 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:25.428704 31310 raft_consensus.cc:697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:25.433952 31144 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:25.434938 31310 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:25.416963 31177 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.438262 31177 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:25.444885 31095 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.446295 31095 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:25.446626 31095 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:25.447441 31095 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.447582 31177 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.448743 31095 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:25.449164 29156 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:25.397140 31225 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:25.452577 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.452493 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:25.453948 31317 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:25.461238 31027 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.461884 31027 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:25.462950 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.464114 31027 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:25.466948 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:25.467427 31170 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:25.457639 31302 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:25.455140 31328 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:11:25.471191 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 75762.6us
I20260709 13:11:25.464790 31317 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:25.469194 31027 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:25.451026 31184 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:25.481216 31184 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:25.481585 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:25.482124 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:25.482477 29003 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:25.482710 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:25.482477 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:25.483820 29001 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:25.483932 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.484300 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.484659 29007 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:25.488881 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 70188us
I20260709 13:11:25.470968 31177 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:25.474080 31317 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:25.495303 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:25.495338 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.496385 31144 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:25.496822 31144 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:25.497213 31144 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:25.498874 29001 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:25.491164 31027 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:25.508015 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.512387 29005 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:25.512763 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.513654 29009 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:25.516834 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.519192 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:25.521523 31249 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:11:25.524935 31216 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:25.525012 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.508069 31027 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.520130 31291 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:25.528951 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.531664 29006 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:25.532171 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:25.535637 29005 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:25.528246 31225 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:25.553872 29007 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:25.555490 29082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:25.556157 29082 raft_consensus.cc:606] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:25.556564 29082 raft_consensus.cc:614] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:25.557592 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:25.559185 29082 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:11:25.559388 29082 pending_rounds.cc:85] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 5
I20260709 13:11:25.559540 29082 pending_rounds.cc:107] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:11:25.559712 29082 raft_consensus.cc:2889] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:25.561923 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 736786us
W20260709 13:11:25.555128 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 172143us
I20260709 13:11:25.568851 31139 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:25.569492 31287 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:25.570801 29051 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:25.571069 31287 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:25.571631 31287 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:25.572194 29050 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:25.582432 29009 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:25.604761 29125 leader_election.cc:400] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4.
I20260709 13:11:25.605158 29125 leader_election.cc:403] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:11:25.609390 31200 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:11:25.611600 31339 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:11:25.609414 29158 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:25.609601 31337 consensus_queue.cc:1059] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:11:25.613336 31337 raft_consensus.cc:3055] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:25.629788 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 265) took 1501 ms. Trace:
I20260709 13:11:25.596297 31144 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.626513 31337 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:25.646502 31337 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:25.648631 31287 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:25.656549 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 252894us
I20260709 13:11:25.657733 28976 leader_election.cc:400] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:25.657845 31337 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:25.659070 31144 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:25.661530 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 86357.5us
W20260709 13:11:25.664737 28908 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51350 (request call id 147) took 19050 ms (client timeout 15000 ms). Trace:
I20260709 13:11:25.668388 31027 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:25.609452 31343 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:25.678128 29006 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:25.682345 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:25.707391 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1086) took 1575 ms. Trace:
I20260709 13:11:25.707742 29682 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:25.720603 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1202) took 1482 ms. Trace:
I20260709 13:11:25.630404 29085 rpcz_store.cc:276] 0709 13:11:24.128703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.827617 (+698914us) service_pool.cc:224] Handling call
0709 13:11:24.827998 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:24.847952 (+ 19954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.847982 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:24.848569 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:11:25.396330 (+547761us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:25.417290 (+ 20960us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.629453 (+212163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.629530 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:25.629567 (+    37us) raft_consensus.cc:1685] finished
0709 13:11:25.629578 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.629732 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":180575,"prepare.queue_time_us":15154,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":213011,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":9644,"spinlock_wait_cycles":1330944,"thread_start_us":602565,"threads_started":2,"wal-append.queue_time_us":610665}
I20260709 13:11:25.708074 29077 rpcz_store.cc:276] 0709 13:11:24.069588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:25.394885 (+1325297us) service_pool.cc:224] Handling call
0709 13:11:25.411610 (+ 16725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.639297 (+227687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:25.639327 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.644732 (+  5405us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:25.644785 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.644915 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.645114 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155636,"raft.queue_time_us":206680,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":16240,"thread_start_us":65281,"threads_started":1}
W20260709 13:11:25.723253 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60890.2us
W20260709 13:11:25.723773 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 203854us
W20260709 13:11:25.730733 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61314.3us
I20260709 13:11:25.732225 29156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "824fc688db264856b025a29c651eec78"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:25.732686 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:25.733084 29156 raft_consensus.cc:606] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:25.733589 29156 raft_consensus.cc:614] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:25.707141 31302 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:25.722519 29155 rpcz_store.cc:276] 0709 13:11:24.238171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.382797 (+144626us) service_pool.cc:224] Handling call
0709 13:11:24.404748 (+ 21951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.710836 (+306088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.707543 (+996707us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54003106d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:25.720522 (+ 12979us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507684096}
I20260709 13:11:25.708530 31216 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:25.741511 29153 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
W20260709 13:11:25.665177 28908 rpcz_store.cc:269] 0709 13:11:06.614070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.620606 (+  6536us) service_pool.cc:224] Handling call
0709 13:11:25.664470 (+19043864us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:21.105555 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:21.106255 (+   700us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:21.106302 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:11:21.106321 (+    19us) tablet.cc:662] Decoding operations
0709 13:11:21.107206 (+   885us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:21.107300 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:21.107332 (+    32us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:11:21.107923 (+   591us) tablet.cc:701] Row locks acquired
0709 13:11:21.107986 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:11:22.119366 (+1011380us) spinlock_profiling.cc:243] Waited 9.68 ms on lock 0x7b2c000f72a8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f4114149a0a 00007f4114149693 00007f4114149631 00007f411587c902 00007f411587c6dd 00007f4119277243 00007f4119276d58 00007f411927b9f1 00007f411927b9a9 00007f411927b939
0709 13:11:22.119474 (+   108us) write_op.cc:270] Start()
0709 13:11:22.119728 (+   254us) write_op.cc:276] Timestamp: P: 1783602681119658 usec, L: 0
0709 13:11:22.119764 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:11:22.170314 (+ 50550us) log.cc:844] Serialized 2496 byte log entry
0709 13:11:23.034454 (+864140us) op_driver.cc:464] REPLICATION: finished
0709 13:11:23.614676 (+580222us) write_op.cc:301] APPLY: starting
0709 13:11:23.622522 (+  7846us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:23.658013 (+ 35491us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:23.658038 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:23.703620 (+ 45582us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:23.795821 (+ 92201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:11:23.804262 (+  8441us) write_op.cc:312] APPLY: finished
0709 13:11:23.804580 (+   318us) log.cc:844] Serialized 53 byte log entry
0709 13:11:23.804759 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:23.804873 (+   114us) write_op.cc:454] Released schema lock
0709 13:11:23.804923 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:23.805410 (+   487us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:23.805459 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":90,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":556918,"apply.run_cpu_time_us":3337,"apply.run_wall_time_us":205001,"num_ops":6,"prepare.queue_time_us":76723,"prepare.run_cpu_time_us":15287,"prepare.run_wall_time_us":1271602,"raft.queue_time_us":224423,"raft.run_cpu_time_us":58937,"raft.run_wall_time_us":224386,"replication_time_us":883244,"spinlock_wait_cycles":22270336,"thread_start_us":499206,"threads_started":4,"wal-append.queue_time_us":202793,"wal-append.run_cpu_time_us":12671,"wal-append.run_wall_time_us":2450447}]]}
I20260709 13:11:25.745277 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:25.746311 29158 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:25.720156 31027 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:25.747902 29154 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:25.749154 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 276) took 1447 ms. Trace:
W20260709 13:11:25.749528 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 175552us
I20260709 13:11:25.760960 29151 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:11:25.751123 29078 rpcz_store.cc:276] 0709 13:11:24.299477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:25.417669 (+1118192us) service_pool.cc:224] Handling call
0709 13:11:25.419072 (+  1403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.419392 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:25.427001 (+  7609us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.746956 (+319955us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.747002 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:25.747044 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.747105 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.747218 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30048,"prepare.queue_time_us":7926,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"thread_start_us":267338,"threads_started":1}
W20260709 13:11:25.761489 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 101485us
I20260709 13:11:25.741767 31291 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.742071 31200 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:25.745797 31144 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 3. Current term is 4.
I20260709 13:11:25.747236 31328 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:25.732275 31353 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:25.773228 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:25.777722 29156 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:25.748344 31312 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:11:25.783111 29079 raft_consensus.cc:606] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:11:25.783521 29079 raft_consensus.cc:614] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:25.784612 29050 leader_election.cc:400] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:25.780493 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:25.780100 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:25.766175 31341 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:25.767601 31061 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:11:25.748680 31302 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:25.789577 31341 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.793912 31351 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:25.794229 31341 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:25.794381 31351 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:25.794795 31351 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:25.794947 31216 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.797910 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:25.801017 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:25.804760 31347 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:25.805637 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:25.806368 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:25.817163 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 274) took 1524 ms. Trace:
I20260709 13:11:25.782989 31249 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:11:25.819039 29156 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:25.817631 29081 rpcz_store.cc:276] 0709 13:11:24.292426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:25.405126 (+1112700us) service_pool.cc:224] Handling call
0709 13:11:25.427274 (+ 22148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.433221 (+  5947us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:25.615050 (+181829us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.816842 (+201792us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.816899 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:25.816938 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.817000 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.817103 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25282,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":969,"raft.queue_time_us":2590,"thread_start_us":282773,"threads_started":2}
W20260709 13:11:25.823093 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 50046.2us
I20260709 13:11:25.790858 31340 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.789065 31144 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:11:25.789590 31291 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:25.830413 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:25.831915 29151 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:25.834419 31356 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:25.835491 31004 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.787930 31353 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:25.836210 31004 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:25.839988 31338 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:11:25.825330 31346 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.842276 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 267) took 1706 ms. Trace:
I20260709 13:11:25.839260 31353 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:25.843726 29006 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:25.846266 29002 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:25.848744 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:25.849020 29151 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:25.849649 29082 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:25.842805 31346 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.847731 29078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:25.852371 29078 raft_consensus.cc:606] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:25.852746 29078 raft_consensus.cc:614] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:25.830893 31200 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:25.829535 31317 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:25.861600 31357 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:11:25.831194 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:25.863090 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:25.864239 31317 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:25.872394 31350 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:25.872938 31350 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:25.865041 31346 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:25.865794 31348 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:25.869203 29007 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:25.869683 29001 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:25.870190 31340 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:25.871235 31287 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:25.864761 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:25.842779 29080 rpcz_store.cc:276] 0709 13:11:24.136143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.935605 (+799462us) service_pool.cc:224] Handling call
0709 13:11:25.372115 (+436510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:25.605965 (+233850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:25.606010 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:25.629200 (+ 23190us) log.cc:844] Serialized 26 byte log entry
0709 13:11:25.782277 (+153077us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:25.782979 (+   702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.783944 (+   965us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:25.842030 (+ 58086us) raft_consensus.cc:1685] finished
0709 13:11:25.842052 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.842207 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182423,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":256807,"threads_started":2,"wal-append.queue_time_us":96588}
I20260709 13:11:26.031144 29154 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:25.973081 29151 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:25.891381 29156 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:11:26.011015 31356 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.039091 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:26.040143 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 175476us
W20260709 13:11:25.938220 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 64854.1us
I20260709 13:11:25.982334 31309 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:26.039258 31095 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:25.886685 31350 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.040853 31340 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:25.942554 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.039762 31356 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:25.887918 29084 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:25.910436 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 102182us
W20260709 13:11:25.934151 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.041065 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:26.045881 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:26.043327 29081 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:26.046653 31349 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:26.042874 31309 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:26.047243 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:26.047552 31095 raft_consensus.cc:2764] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:11:26.048856 31349 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:26.047854 31356 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.047952 29154 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.050016 31095 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:26.051674 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.053036 29154 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.040663 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.054128 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.052625 31356 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:26.054708 31095 consensus_queue.cc:1059] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:26.055171 31356 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:26.056087 31027 raft_consensus.cc:3055] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:26.056533 31027 raft_consensus.cc:740] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:26.051363 31309 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.057201 31169 consensus_queue.cc:1059] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:26.057431 31027 consensus_queue.cc:260] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.058308 31309 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.040697 29156 pending_rounds.cc:85] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 5
I20260709 13:11:26.053526 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.059151 29156 pending_rounds.cc:107] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:11:26.052199 31349 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:11:26.059480 29156 raft_consensus.cc:2889] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:26.060127 30921 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:26.060487 30921 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:26.060847 29156 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:26.061328 30921 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:26.063608 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.054337 29151 raft_consensus.cc:2393] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:25.827836 31328 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.065045 31177 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.066170 31346 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.068516 31291 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.069715 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:26.073128 31340 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:26.073123 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.074893 31341 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:26.080681 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.075192 31351 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.081938 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:26.092355 29156 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.093212 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.093801 31351 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.094941 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.095595 29151 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:26.101594 31350 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.104002 31350 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:26.113960 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51874.3us
I20260709 13:11:26.118803 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:26.120445 31350 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:26.122448 31350 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:26.122828 31350 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:26.123231 31350 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:26.076391 30921 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:26.037773 31348 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:26.128981 31349 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:26.129200 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 224828us
I20260709 13:11:26.131223 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:26.129698 29711 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:26.133282 31349 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.139223 31349 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.128268 31346 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.135514 31340 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.140908 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1081) took 2072 ms. Trace:
I20260709 13:11:26.141299 29084 rpcz_store.cc:276] 0709 13:11:24.068241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.714877 (+646636us) service_pool.cc:224] Handling call
0709 13:11:24.715248 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:26.043445 (+1328197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:26.140659 (+ 97214us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000c99d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:26.140827 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2696122112}
I20260709 13:11:26.141994 31346 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.142293 31340 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.133936 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:26.143733 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:26.144623 29122 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:26.142683 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.129405 31366 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:11:26.145748 31027 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:26.056598 31356 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:26.146649 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 413995us
I20260709 13:11:26.147383 31356 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.147485 31371 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:26.147876 31266 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:26.148352 31266 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.148643 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:26.149374 31266 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.150300 31216 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:26.150683 31266 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.150864 31216 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.151594 31341 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.152207 31341 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.151847 31216 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.152884 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.152817 31341 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.153309 31144 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:26.153640 29009 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:26.149791 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:26.155190 31351 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:26.156175 31371 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.161835 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:26.163272 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 245) took 1381 ms. Trace:
I20260709 13:11:26.130137 31260 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:26.130296 31348 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.182821 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:26.187865 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.191062 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:26.191675 29151 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.193938 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.195441 29680 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:26.182643 31210 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:26.196748 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:26.200511 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 50835.6us
I20260709 13:11:26.183655 31351 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:26.203096 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.184582 31260 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:26.204025 29004 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:26.186115 31366 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:26.185727 31144 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.204388 31353 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:26.216323 31371 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.216948 31353 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.217424 31353 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.219890 31364 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:26.220434 31364 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.220538 29009 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:26.220796 31364 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.221796 29007 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:26.206429 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:26.183478 29157 rpcz_store.cc:276] 0709 13:11:24.782332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.784919 (+  2587us) service_pool.cc:224] Handling call
0709 13:11:24.824415 (+ 39496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:25.456945 (+632530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:25.456974 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:25.466184 (+  9210us) log.cc:844] Serialized 26 byte log entry
0709 13:11:25.515949 (+ 49765us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:25.515987 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:26.162864 (+646877us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54004010d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:26.162911 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:26.162954 (+    43us) raft_consensus.cc:1685] finished
0709 13:11:26.162966 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:26.163202 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3307,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":5649,"spinlock_wait_cycles":1444022528,"thread_start_us":49356,"threads_started":1,"wal-append.queue_time_us":79500}
I20260709 13:11:26.212847 29150 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.225916 29717 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.238350 29150 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:26.206584 31351 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:26.242463 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:26.243551 31341 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:26.254578 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 135900us
I20260709 13:11:26.237965 31366 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.270859 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:26.259127 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43527.4us
I20260709 13:11:26.260536 31184 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:26.245510 31266 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:11:26.265399 29004 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:26.268303 31144 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.240793 31302 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.259810 31341 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.291699 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.287864 31249 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:11:26.292644 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.243920 31362 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:11:26.257045 31317 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:11:26.295125 31249 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.299865 31260 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.301180 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:26.303552 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:26.308244 29156 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.294303 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1076) took 2121 ms. Trace:
I20260709 13:11:26.308563 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:26.311187 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.312942 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.315342 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.313831 31362 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:26.304171 31341 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.308736 29083 rpcz_store.cc:276] 0709 13:11:24.066872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.653954 (+587082us) service_pool.cc:224] Handling call
0709 13:11:24.654576 (+   622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.655114 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:24.768861 (+113747us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.367217 (+598356us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.569129 (+201912us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b18003a5fa0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f411971558f 00007f4112754571 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd
0709 13:11:26.137358 (+568229us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b18003a5fa0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:11:26.182392 (+ 45034us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:26.182439 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:26.182482 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:26.184144 (+  1662us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:26.188578 (+  4434us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8926,"prepare.queue_time_us":135327,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":93,"raft.queue_time_us":202023,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":10512,"spinlock_wait_cycles":13562496,"thread_start_us":637558,"threads_started":3,"wal-append.queue_time_us":631622}
W20260709 13:11:26.351991 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 48603.4us
I20260709 13:11:26.352726 29716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 381) took 2297 ms. Trace:
I20260709 13:11:26.302042 31366 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.353260 29716 rpcz_store.cc:276] 0709 13:11:24.051424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.117950 (+ 66526us) service_pool.cc:224] Handling call
0709 13:11:24.131694 (+ 13744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.383833 (+1252139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:26.326904 (+943071us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400576f54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:26.348925 (+ 22021us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2861819776}
I20260709 13:11:26.352522 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:26.355039 31260 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:26.362733 31184 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.367264 31348 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.352609 31385 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:26.364302 29001 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:26.384624 31027 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:26.386598 31027 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.387245 31348 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.390264 31027 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.394269 29156 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
W20260709 13:11:26.401108 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51690us
I20260709 13:11:26.376802 31266 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:26.408223 31184 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.413709 29125 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:26.415092 31362 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:26.415841 31362 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:26.416735 31362 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:26.417049 31362 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.417872 31362 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.421182 31341 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.419298 31362 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:26.416026 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:26.428316 29717 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:26.376417 31353 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.360993 31249 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:26.436234 31389 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:11:26.435925 31247 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:26.411947 31266 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:26.442358 29159 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:26.445040 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.454355 31351 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.459620 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:26.447330 31392 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:11:26.469838 29157 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.473071 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 257) took 2438 ms. Trace:
I20260709 13:11:26.473628 29010 rpcz_store.cc:276] 0709 13:11:24.034462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.340655 (+306193us) service_pool.cc:224] Handling call
0709 13:11:24.378842 (+ 38187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:24.831641 (+452799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.831690 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:26.152117 (+1320427us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b2c00000f48. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:26.152515 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 13:11:26.178932 (+ 26417us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:26.178971 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:26.184549 (+  5578us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:26.472500 (+287951us) raft_consensus.cc:1685] finished
0709 13:11:26.472531 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:26.472832 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170,"raft.queue_time_us":10386,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":19940,"spinlock_wait_cycles":1216225152,"thread_start_us":429713,"threads_started":2,"wal-append.queue_time_us":96683}
I20260709 13:11:26.411042 31364 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.477885 31364 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.478621 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.478895 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.479852 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.480110 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.480374 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.480736 29085 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.480962 29084 raft_consensus.cc:2376] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6. Re-sending same reply.
I20260709 13:11:26.480893 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.481645 29083 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:26.481693 29082 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:26.443614 31266 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:26.463528 31390 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:11:26.494611 29001 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:26.498032 29051 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:26.498212 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:26.499081 29711 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:26.499261 31287 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:26.499831 31287 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:26.500801 31287 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:26.505132 31360 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.691s	user 0.003s	sys 0.000s
I20260709 13:11:26.507292 29051 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:26.507499 31144 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.507951 31144 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.508294 31341 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:26.508764 31341 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.509107 31341 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.510061 31144 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:26.510668 31144 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:26.510980 31144 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.511745 31144 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.512837 31144 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.518918 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44418 (request call id 78) took 2334 ms. Trace:
I20260709 13:11:26.519320 29079 rpcz_store.cc:276] 0709 13:11:24.198390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:25.420296 (+1221906us) service_pool.cc:224] Handling call
0709 13:11:26.518713 (+1098417us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400148a54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583b853 00007f411a80960a 00007f4113d8f7d4 00007f4113d8f762 00007f4113d8f691 00007f4113d8f60c 00007f4113d8e8c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:26.518824 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807116160}
I20260709 13:11:26.523320 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.523913 29010 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.528522 31027 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.529266 31027 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.530959 29010 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:26.532325 31337 consensus_queue.cc:1059] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:26.533572 31310 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:26.533983 31310 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:26.534426 31388 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:11:26.534467 29680 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:26.534776 31310 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.536084 31310 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.537127 29079 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:11:26.536084 31403 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:26.538937 31403 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:26.540022 31403 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:26.542908 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.495474 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:26.546507 29716 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.545503 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:26.468894 31371 raft_consensus.cc:2764] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:11:26.531701 31027 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:26.584453 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 146291us
I20260709 13:11:26.549830 29082 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:26.588369 29079 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:11:26.548679 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.549366 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.549062 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:26.600068 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.549083 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:26.612078 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.618533 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.618676 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.619045 29084 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:26.619213 29157 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:26.545008 31351 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:26.621707 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 205807us
W20260709 13:11:26.622426 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 73873.6us
W20260709 13:11:26.623283 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.620661 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.490602 31350 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.639075 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:26.621194 31139 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.643070 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.647505 29082 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:26.648365 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.648990 29076 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:26.650872 29157 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:26.646957 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.652127 29078 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.655956 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.656865 29084 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.659945 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.667462 29717 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:26.673781 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:26.675113 29082 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:26.676362 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.674546 29714 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:26.677325 29050 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:26.679800 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.683750 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.684330 29081 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.685897 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.686684 29159 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.681926 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:26.665849 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:26.690537 29154 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:26.697404 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:26.697526 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60024.2us
I20260709 13:11:26.698344 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.699256 29152 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:26.650527 31139 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.700964 29153 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:26.682369 31247 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:26.702668 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.703864 29076 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:26.700239 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:26.707228 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:26.716794 31408 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:11:26.719563 31404 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:11:26.698714 31385 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.751284 31385 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.726984 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.729949 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.706565 29151 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:26.731416 29157 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:26.702905 31287 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:26.684772 31367 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:11:26.753747 31385 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.753983 29001 raft_consensus.cc:2368] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:26.736310 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.754436 31302 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.754801 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.730960 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.755080 31004 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.755409 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.755019 31302 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.756007 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.756486 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.757009 29002 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.757077 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.757507 31302 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.757834 29005 raft_consensus.cc:2393] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.758659 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.737056 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.760583 29005 raft_consensus.cc:2393] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:26.760782 29010 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:26.761284 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.761487 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.762084 29010 raft_consensus.cc:2368] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:26.762320 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.762739 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.762861 29005 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:26.763648 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.739125 31249 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.767252 29683 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:26.768563 31309 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:26.771035 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:26.773484 31403 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:26.773910 31403 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.774268 31403 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:26.774380 31356 consensus_queue.cc:1059] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.775261 31356 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:26.775964 31356 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:26.776662 31356 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:26.738485 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.779359 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.780372 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.780525 31352 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:26.781070 31352 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.781412 31348 raft_consensus.cc:3055] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:26.781819 31348 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:26.782179 31352 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.782666 31348 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.783581 31352 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.783674 31247 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.739127 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.784711 29004 raft_consensus.cc:2368] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:26.784866 31247 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.785749 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.786005 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.786604 29159 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:26.739488 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.788015 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.788818 29003 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:26.739769 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.789592 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.790217 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.790822 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:26.791079 31416 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:11:26.791189 31309 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:26.740449 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.792340 31027 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:26.792735 31027 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:26.793063 29005 raft_consensus.cc:2368] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
W20260709 13:11:26.793133 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65668.9us
I20260709 13:11:26.793556 31027 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.794512 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:26.794809 31027 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:26.795327 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.747648 31400 consensus_queue.cc:1059] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:26.795538 31210 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:26.748939 31353 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.796036 31400 raft_consensus.cc:3055] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:26.795382 31287 consensus_queue.cc:1059] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:26.796463 31400 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:26.796582 31353 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.797163 31353 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.797279 31400 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.751646 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:26.784682 29155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:26.790817 29006 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:26.798585 29155 raft_consensus.cc:606] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:26.798987 29155 raft_consensus.cc:614] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:26.747164 31368 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:26.786844 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.799700 31368 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:26.800137 29004 raft_consensus.cc:2368] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:26.797094 31287 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:26.800750 31287 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:26.749526 31350 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.738819 31139 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:26.801783 31287 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.799552 31353 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:26.800140 31368 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:26.780925 29007 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:26.825141 29083 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:26.817643 29084 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:26.806797 29008 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
W20260709 13:11:26.810035 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 107165us
W20260709 13:11:26.812320 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.812772 29085 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
W20260709 13:11:26.815723 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.817185 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:26.823477 29716 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:26.817495 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.829926 29080 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:26.816856 31025 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:26.830700 31025 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:26.814838 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.830992 29082 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:11:26.805450 31312 consensus_queue.cc:1175] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "befda4a6c8344c26b0f139e4e1686b3b" responder_term: 5 status { last_received { term: 5 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 5 index: 5 } }
I20260709 13:11:26.832119 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.783856 31348 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:26.818171 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 66627.4us
I20260709 13:11:26.839218 29122 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:26.844276 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.831578 31025 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.833319 29081 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:26.833563 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:26.838114 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.821775 31385 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:26.818087 31350 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:26.817236 31287 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.850965 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.851725 30921 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:26.827356 29711 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:26.847252 31317 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:11:26.854080 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.857113 29050 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:26.857208 29007 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:26.858804 31368 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.861080 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:26.865113 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.884447 29682 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:26.887245 29008 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:26.865917 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:26.852665 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:26.888339 29083 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:26.873911 29151 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:26.870373 29003 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:26.853812 29048 leader_election.cc:400] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:26.889617 29048 leader_election.cc:403] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:11:26.872784 29078 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:26.877434 29010 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:26.868184 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:26.892179 29010 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:26.893164 29003 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:26.866652 31309 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:26.866782 31385 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:26.897145 31340 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:26.899382 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:26.902820 29716 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:26.902820 29008 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:26.905640 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.906481 29009 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:26.908138 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.909765 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:26.921496 29010 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:11:26.931907 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 70886us
I20260709 13:11:26.934958 29009 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:26.895900 31309 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.906565 31302 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:26.939200 29002 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:26.941359 31210 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.942018 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:26.928224 31340 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
W20260709 13:11:26.946126 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 172439us
I20260709 13:11:26.914459 31317 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:26.900972 31266 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.936132 31362 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:26.895213 31341 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:26.943796 31309 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:26.939574 31025 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:26.943444 31302 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:27.000603 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 146948us
I20260709 13:11:26.986503 31302 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:26.949780 31210 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.007318 31170 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.007843 31170 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.009689 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:27.010509 31170 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:27.017673 31266 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:27.020937 31428 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.021631 31428 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.029711 31341 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:27.034276 31317 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:26.944510 31385 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.044551 29082 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
W20260709 13:11:27.048158 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.048816 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.049185 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 113253us
W20260709 13:11:27.051118 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45467.8us
I20260709 13:11:27.051378 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:26.998445 31350 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:27.054483 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:27.057642 31387 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:27.054804 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:27.054857 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:27.061362 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.060282 31350 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:27.055210 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 45603.6us
I20260709 13:11:27.066401 31431 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:27.069885 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:27.056003 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.073899 29010 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:27.082976 31409 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.083159 31403 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.084966 31309 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.086089 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:27.086638 29002 raft_consensus.cc:2376] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5. Re-sending same reply.
W20260709 13:11:27.389701 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 323071us
W20260709 13:11:27.234601 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.169778 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.391605 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.087558 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:27.087610 29155 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:11:27.053640 31385 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:27.388994 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.092427 29001 raft_consensus.cc:2410] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:27.392875 29002 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:27.396034 29079 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:27.336151 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.392724 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 506671us
I20260709 13:11:27.088716 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:27.218477 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 271496us
W20260709 13:11:27.386873 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 332096us
I20260709 13:11:27.257205 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:27.091837 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.400796 29076 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:27.402196 29004 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:27.402212 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 346239us
I20260709 13:11:27.056792 31249 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.405730 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:27.410544 28975 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:27.417472 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.070214 31170 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:27.255903 31403 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:27.413931 28975 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:27.412974 29048 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.275704 31287 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:27.256513 31428 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:27.087024 31309 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:27.166548 31341 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:27.090353 31387 consensus_queue.cc:1243] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:11:27.279369 29709 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:27.398875 31210 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:27.440891 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 44763.5us
I20260709 13:11:27.462881 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:27.424907 31356 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:27.468187 31210 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:27.467916 29153 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:27.469509 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:27.469208 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:27.473351 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:27.474097 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:27.474537 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:27.475215 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:27.425936 31348 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:27.483748 29004 raft_consensus.cc:2368] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:27.486299 29079 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:27.385910 31437 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:27.444631 31400 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:27.466768 31439 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:27.510174 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:27.511348 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:27.511991 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:27.517720 29006 raft_consensus.cc:2368] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:27.519333 29009 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:27.523644 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:27.527060 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 92369.7us
W20260709 13:11:27.527216 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 126475us
I20260709 13:11:27.432582 31249 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.530228 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:27.530807 29712 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:27.531253 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.531490 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:27.476402 31434 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:11:27.517086 31437 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.516827 31348 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.537696 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:27.533568 31433 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:27.542768 29711 raft_consensus.cc:2410] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:11:27.543820 31436 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:11:27.544560 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 75478.8us
I20260709 13:11:27.546243 29710 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:11:27.547955 31410 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.857s	user 0.005s	sys 0.015s
I20260709 13:11:27.478251 31341 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.530629 31439 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:27.533797 31356 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:27.533320 31139 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.535858 31317 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.568689 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:27.555629 31004 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:27.538898 31437 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:27.573784 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:27.538722 31348 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:27.586742 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 173885us
I20260709 13:11:27.583453 31210 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:27.554349 31302 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.607455 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.595572 31004 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:27.608870 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:27.609578 29710 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:27.614456 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:11:27.597239 31025 consensus_meta.cc:287] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.003s
W20260709 13:11:27.633230 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.643507 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 132245us
I20260709 13:11:27.644739 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:27.645764 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.648458 29082 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:27.568431 31441 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:11:27.650408 29717 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:27.587560 31387 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:11:27.654780 29082 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
W20260709 13:11:27.656582 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 57304.9us
W20260709 13:11:27.655290 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 127548us
I20260709 13:11:27.657437 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.658581 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:27.587805 31139 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.591946 31317 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.663767 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.661734 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.664878 29080 raft_consensus.cc:2410] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:11:27.665041 29081 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:27.659932 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.665819 29077 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:27.667351 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:27.668520 29083 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:27.606135 31434 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:11:27.663582 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:27.635272 31351 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:11:27.673091 31341 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:27.673638 31341 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:27.674222 31341 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:27.675503 29710 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:27.676494 31317 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:27.677206 31302 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:27.656343 31447 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:27.660619 31025 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:27.682865 31442 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:27.697661 31139 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:27.701409 29712 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:27.703504 29076 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:27.706248 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:27.707050 29083 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:27.707520 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.633896 31210 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.659571 31439 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.716482 29155 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:11:27.719174 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60645.5us
I20260709 13:11:27.713212 29683 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:27.721980 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.723187 31139 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:27.689726 31434 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:27.724640 31444 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:11:27.609916 31004 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:27.727078 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 208146us
I20260709 13:11:27.690933 31442 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.726940 29007 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:11:27.740933 29007 pending_rounds.cc:85] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 2
I20260709 13:11:27.741173 29007 pending_rounds.cc:107] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:11:27.741425 29007 raft_consensus.cc:2889] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:27.721341 31367 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:27.724725 31210 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:27.738648 29076 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:11:27.738943 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:27.723712 31443 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:11:27.741735 29007 pending_rounds.cc:107] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:11:27.750797 29007 raft_consensus.cc:2889] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:27.748737 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.726228 31341 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:27.755566 31341 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.756268 31445 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:27.713222 31433 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:27.754762 31441 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:27.730994 31139 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:27.769389 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 143606us
I20260709 13:11:27.771513 29010 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:27.774883 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:27.775619 29010 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:27.776302 31170 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:27.729583 31287 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:27.743510 31442 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:27.784783 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:27.788486 29122 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:27.794432 29153 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:27.796177 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.796281 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.769503 31139 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:27.799121 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 50494.8us
W20260709 13:11:27.800897 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 243104us
I20260709 13:11:27.751402 31434 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:27.806216 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:27.807215 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.813254 29083 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:11:27.813462 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 71045.2us
I20260709 13:11:27.795660 31445 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:27.816521 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1610) took 1009 ms. Trace:
I20260709 13:11:27.798758 31170 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:27.890432 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 72810.6us
I20260709 13:11:27.818938 29008 rpcz_store.cc:276] 0709 13:11:26.806736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:26.909761 (+103025us) service_pool.cc:224] Handling call
0709 13:11:26.932956 (+ 23195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:26.933182 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:26.954456 (+ 21274us) log.cc:844] Serialized 14 byte log entry
0709 13:11:27.676416 (+721960us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b180034fa20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:11:27.816156 (+139740us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:27.816209 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:27.816257 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:27.816318 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:27.816451 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14953,"prepare.queue_time_us":20980,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":116905,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":6815,"spinlock_wait_cycles":6650112,"thread_start_us":139626,"threads_started":1}
I20260709 13:11:27.875866 29076 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:27.894960 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.896596 29051 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:27.782516 31450 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:11:27.902827 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:27.903544 29004 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:27.797920 31433 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.899695 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:27.899894 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 99279.4us
I20260709 13:11:27.901772 29083 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:27.823354 31385 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:27.899073 31443 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:11:27.905310 29001 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:27.825714 31170 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.920758 29048 leader_election.cc:400] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:27.924803 29048 leader_election.cc:403] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:11:27.924963 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.909955 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 108404us
I20260709 13:11:27.895522 31341 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:27.923024 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:27.933001 29083 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:27.933985 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:27.934959 29710 raft_consensus.cc:2393] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:27.934046 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:27.932219 31170 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:27.948338 31170 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.949273 31027 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:27.949790 31027 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:27.950881 31170 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:27.950830 31027 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.953080 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:27.960992 31249 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:27.962630 31441 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
W20260709 13:11:27.965708 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.968001 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 47423.7us
I20260709 13:11:27.968374 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:27.968984 29001 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:27.973138 29005 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:27.973372 31434 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:27.974957 31249 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:27.976759 31437 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.978861 31437 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:27.980356 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.980641 29711 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:27.980998 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:27.982867 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:27.983523 29710 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:27.984871 31352 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:27.985803 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.986162 31317 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.987526 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:27.988713 29155 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:27.989625 29151 raft_consensus.cc:1240] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:27.990046 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:27.991356 31309 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:27.986750 31317 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:27.993583 29150 raft_consensus.cc:1813] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:27.994381 29150 raft_consensus.cc:2410] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:11:27.988955 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:27.997249 29002 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:27.968990 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:27.998796 29048 leader_election.cc:400] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:27.999039 29048 leader_election.cc:403] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:11:28.001279 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.002641 29157 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:28.004161 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.005005 29157 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:28.005820 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:28.007002 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:28.007604 29157 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:28.012534 29001 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:28.015228 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:28.016204 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63198.4us
I20260709 13:11:28.017197 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:27.904237 31453 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:27.906428 31448 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:28.025432 29154 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:28.032675 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.026120 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.027266 29150 raft_consensus.cc:2410] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:11:28.027683 29152 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:28.028671 29156 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:27.902017 31287 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 13:11:28.029631 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:28.039108 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.045151 29155 raft_consensus.cc:2410] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:28.048007 29158 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:28.015087 31352 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.041510 29151 raft_consensus.cc:2368] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5.
I20260709 13:11:28.038657 29154 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:11:28.038091 31454 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:28.055852 29005 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:11:28.057337 29005 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:11:28.059036 31463 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:11:28.059898 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.064014 29077 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:28.066974 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:28.065961 29085 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:28.072872 29001 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:28.073058 29157 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:28.073623 29004 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:28.039621 29008 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:28.030524 31437 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
W20260709 13:11:28.044251 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45538.4us
I20260709 13:11:28.079052 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:28.019832 31341 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:28.041548 29122 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:28.043982 31027 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:28.065256 29708 consensus_meta.cc:287] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.003s
W20260709 13:11:28.087731 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.088006 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.054261 31448 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:28.085446 29048 connection.cc:256] RPC call timeout handler was delayed by -1.1134858479999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 13:11:28.092451 28975 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:28.110683 29005 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:28.118228 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:28.123107 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.063629 31139 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.067117 31432 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:11:28.089778 31170 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:28.089540 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:28.147691 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:28.127635 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1606) took 1320 ms. Trace:
I20260709 13:11:28.148691 31367 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:28.075265 29002 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:11:28.149128 31452 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:28.149492 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 1997) took 1093 ms. Trace:
I20260709 13:11:28.073565 31352 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.081314 31341 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:28.092133 29708 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:28.149260 31428 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:28.149853 31452 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:28.126036 29085 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:28.119228 31437 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.149943 29002 rpcz_store.cc:276] 0709 13:11:27.056143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:27.988688 (+932545us) service_pool.cc:224] Handling call
0709 13:11:28.149435 (+160747us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:28.060006 31348 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.124321 31385 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.089771 31441 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:28.111167 31455 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:11:28.111778 31351 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:11:28.158068 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.154292 31438 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:28.159089 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:28.143148 31317 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:28.150153 31341 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:28.164938 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 183994us
I20260709 13:11:28.177018 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:28.150704 31454 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:28.148542 29003 rpcz_store.cc:276] 0709 13:11:26.801150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:26.907858 (+106708us) service_pool.cc:224] Handling call
0709 13:11:26.908274 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:26.987643 (+ 79369us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:26.996771 (+  9128us) log.cc:844] Serialized 14 byte log entry
0709 13:11:27.476620 (+479849us) log.cc:844] Serialized 14 byte log entry
0709 13:11:27.476790 (+   170us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:27.506772 (+ 29982us) log.cc:844] Serialized 26 byte log entry
0709 13:11:27.506932 (+   160us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:27.506980 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:27.507070 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:27.956065 (+448995us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b18003ef5c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:11:28.121287 (+165222us) raft_consensus.cc:1685] finished
0709 13:11:28.121322 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:28.121505 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85032,"prepare.queue_time_us":8443,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":2316,"raft.queue_time_us":133681,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":165,"spinlock_wait_cycles":13633024,"thread_start_us":486870,"threads_started":2,"wal-append.queue_time_us":588448}
I20260709 13:11:28.167234 29077 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:28.090164 31456 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.153532 31431 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:28.160141 31437 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.188073 31456 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.189499 31437 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:28.174821 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:28.156527 31170 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:28.180330 31438 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:28.194883 31438 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.181599 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.198225 29080 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:28.183493 29716 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:28.183156 31442 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.184976 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.183193 31139 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:28.190151 31456 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:28.169533 29085 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:28.193171 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 1989) took 1446 ms. Trace:
I20260709 13:11:28.193380 31431 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:28.193459 31348 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:28.193650 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:28.259754 31431 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:28.259583 29007 rpcz_store.cc:276] 0709 13:11:26.746275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:26.891500 (+145225us) service_pool.cc:224] Handling call
0709 13:11:26.891790 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:27.767106 (+875316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:28.192923 (+425817us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54003aaa54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:28.193087 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1916300672}
W20260709 13:11:28.260305 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 66711.5us
I20260709 13:11:28.260488 31431 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:28.192510 31170 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:28.247282 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 70337.3us
I20260709 13:11:28.251922 31139 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:28.209328 31317 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:28.233860 31260 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:28.150529 31421 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:11:28.264663 31456 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:28.180183 31350 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:28.264708 31139 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.265270 31456 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:28.265682 31260 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:28.263796 29076 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:28.267755 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:28.262396 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:28.269626 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:28.265727 31456 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:28.266506 31350 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:28.240144 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.553s (SENT) [suppressed 28 similar messages]
I20260709 13:11:28.269560 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:28.272396 31467 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:11:28.272400 31455 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:28.272734 31449 raft_consensus.cc:2764] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:11:28.273773 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:28.270432 31317 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:28.270428 31433 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:28.205179 29077 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.167065 31385 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.291534 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:28.282629 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 271) took 1026 ms. Trace:
I20260709 13:11:28.283818 29680 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:28.284719 31461 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:28.276427 31350 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.149808 31367 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.292232 29153 rpcz_store.cc:276] 0709 13:11:27.256654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:27.256843 (+   189us) service_pool.cc:224] Handling call
0709 13:11:28.282445 (+1025602us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b54002b52d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:28.282574 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160302336}
I20260709 13:11:28.243074 31352 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:28.302726 31472 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:11:28.236959 31448 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.304272 31312 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:28.304122 31317 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:28.290911 31433 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:28.304708 31249 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:28.241580 31452 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.295717 31455 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:11:28.323104 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 83002.3us
I20260709 13:11:28.321177 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:28.320277 29077 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:28.319422 31351 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.319146 31266 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:28.333278 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:28.322871 31139 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:28.323779 31249 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:28.324306 31442 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:28.327144 31312 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:28.344470 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:28.343647 29709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 352) took 1693 ms (client timeout 2144 ms). Trace:
I20260709 13:11:28.349893 31480 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:28.350315 31480 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:28.350675 31480 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.326061 31352 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:11:28.337706 31170 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:28.345022 31249 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:28.361370 31442 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:28.335379 31340 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:11:28.334537 31437 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:11:28.335740 31351 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.352841 31287 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:28.339015 31266 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.361606 31170 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:28.338941 31453 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 13:11:28.382855 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.384006 29159 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:28.387609 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.388221 29002 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:28.388307 29085 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:11:28.384754 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:28.389595 29007 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.393198 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.394632 28975 leader_election.cc:400] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5.
I20260709 13:11:28.396770 29085 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:28.395761 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:28.349704 29709 rpcz_store.cc:269] 0709 13:11:26.646297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:26.648564 (+  2267us) service_pool.cc:224] Handling call
0709 13:11:28.333202 (+1684638us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400650254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:28.337064 (+  3862us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246146944}
W20260709 13:11:28.398815 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 77641.7us
I20260709 13:11:28.400897 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:28.403707 29152 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:28.404614 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:28.405572 29156 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:28.359280 31341 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.406067 29158 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.406921 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:28.378751 31287 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:28.408802 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:28.361774 31455 raft_consensus.cc:697] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:28.409359 31210 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:28.413269 31210 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:28.415674 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.417721 29078 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:28.409739 29085 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:28.409662 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.397557 31348 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.421846 31348 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.419365 31312 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.425308 29007 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:28.426090 29158 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:28.427680 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.428359 31170 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.429369 31170 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:28.431169 31249 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.431977 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:28.432415 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.416473 31287 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:28.432987 29081 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:28.433915 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.402148 31435 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.436362 31435 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.439488 31266 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:28.441406 31341 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:28.441946 29082 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:28.442863 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.443428 29082 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:28.444439 29076 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:28.446918 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:28.450234 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.451010 29080 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:28.452672 29085 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:11:28.408241 31441 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:28.453090 29085 pending_rounds.cc:85] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 2
I20260709 13:11:28.466499 29085 pending_rounds.cc:107] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:11:28.454471 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:28.409489 31449 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:28.455919 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.377401 31433 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.456365 31351 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:28.459591 31341 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:28.453820 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.466872 29080 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:28.472342 29085 raft_consensus.cc:2889] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:28.482442 31442 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:28.464504 31455 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.420310 31472 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.501636 31482 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:28.516577 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.524657 29085 raft_consensus.cc:2368] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5.
I20260709 13:11:28.526356 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:28.533090 29081 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:28.533882 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.536442 31249 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:28.538422 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:28.538769 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.508440 31438 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:28.516477 31351 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:28.539244 29085 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:28.510082 31341 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.560289 31341 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:28.548823 31449 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.569159 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.527208 31472 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.579797 29154 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:28.580796 29159 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:28.589500 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:28.593202 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.594205 29157 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:28.601169 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.551090 31438 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:28.610016 29081 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:28.611208 29077 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:28.612879 31368 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:28.603128 31004 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:11:28.602975 31439 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:11:28.629696 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.630093 29050 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:28.614992 29083 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:28.630682 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:28.615612 29155 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:28.621502 29078 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:28.560976 31480 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.635001 29077 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:28.639886 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.576851 31456 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:28.649480 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 202601us
I20260709 13:11:28.656919 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:28.659420 29157 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:11:28.663262 29159 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:28.664084 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:28.664804 29009 raft_consensus.cc:2376] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4. Re-sending same reply.
I20260709 13:11:28.638598 31442 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:28.668946 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.643178 31428 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:28.671994 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.687322 29005 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:28.673202 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.675447 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:28.688002 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:28.676334 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.676631 31385 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:28.677786 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:28.683537 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.690366 29004 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:28.690600 29077 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:28.693334 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.693862 29077 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:28.696729 29005 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:28.576455 31449 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.639228 31341 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:11:28.666407 31350 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.636389 31448 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:11:28.664305 31437 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.717692 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:28.741240 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:28.723208 31356 consensus_queue.cc:1059] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:28.746411 29683 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:28.661499 31340 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:11:28.727887 31435 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:28.741245 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44945.3us
I20260709 13:11:28.721668 31480 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:28.667678 31439 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:28.686559 31428 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:28.767102 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.774482 29079 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.762876 30921 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.697881 31352 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:28.832593 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 385601us
I20260709 13:11:28.723642 31431 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.764066 31341 consensus_queue.cc:1243] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:11:28.856139 28976 leader_election.cc:400] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5.
I20260709 13:11:28.863389 28976 leader_election.cc:403] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:11:28.840431 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 431353us
I20260709 13:11:28.855722 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:28.851310 28956 auto_rebalancer.cc:893] Removing replace marker failed: Replica befda4a6c8344c26b0f139e4e1686b3b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:28.876720 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:28.800518 31464 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.817831 31448 consensus_queue.cc:1243] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:11:28.817548 31451 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:11:28.877650 28956 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:11:28.878159 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.879021 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:28.885653 29157 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:28.885798 29010 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
W20260709 13:11:28.886422 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.745965 31350 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:28.872927 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 197529us
I20260709 13:11:28.765528 31437 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.886574 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.879218 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:28.887861 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:28.851018 30921 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:28.885450 28956 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:11:28.930781 30921 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:28.932849 31437 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:28.888489 31352 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:28.933749 31352 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:28.934020 29157 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:28.934122 31352 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:28.935372 29157 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:11:28.937566 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:28.939687 31437 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:28.851547 28907 catalog_manager.cc:5697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 3, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 3 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:28.864348 31438 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:28.873623 31454 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.948899 29079 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
W20260709 13:11:28.883503 31485 log.cc:927] Time spent T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:11:28.950652 29083 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:11:28.948562 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.951182 29082 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:28.830447 31439 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.820519 31287 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:29.035799 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 96372.7us
I20260709 13:11:29.036800 29125 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:29.035733 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 156590us
I20260709 13:11:29.039657 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:29.030144 28907 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 3, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:29.104031 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.118245 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:29.037832 31451 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:29.119462 28907 catalog_manager.cc:5697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 4, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 4 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:28.850876 31428 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:28.979848 31356 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5.
I20260709 13:11:28.980732 31437 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:29.150153 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 110552us
I20260709 13:11:28.889092 31464 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.049873 31461 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:29.159699 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:29.157022 29048 leader_election.cc:400] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:29.080997 31433 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:29.175063 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:29.179854 29002 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:29.179486 28907 catalog_manager.cc:5697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 4, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 4 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:29.180191 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 301292us
I20260709 13:11:29.132563 31341 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:29.178712 29048 leader_election.cc:403] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:11:29.185088 29082 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:29.185693 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:29.186764 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:29.187319 29006 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:29.057687 31448 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.112s
I20260709 13:11:29.192168 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:29.186312 29083 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:29.170047 31442 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:29.203794 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46955.8us
I20260709 13:11:29.205229 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:29.179816 31456 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:29.257773 29714 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:29.037676 31454 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:29.195796 31502 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:11:29.262617 28907 catalog_manager.cc:5697] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 3, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.262024 29076 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:11:29.191380 31356 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:29.267745 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 62597.8us
I20260709 13:11:29.271251 31445 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:11:29.284628 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 97632.5us
W20260709 13:11:29.287828 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 213297us
I20260709 13:11:29.151996 31510 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.129s
I20260709 13:11:29.300212 31495 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.300808 31495 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.300627 28907 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 2, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.302488 28907 catalog_manager.cc:5697] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:29.258347 31341 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5.
I20260709 13:11:29.308377 29006 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:29.296689 29048 leader_election.cc:400] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
W20260709 13:11:29.303848 31488 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:11:29.264094 31515 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.265283 31302 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.282181 31356 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.317337 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:29.296146 31448 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.283470 31482 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:29.317135 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:29.182318 31461 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.322790 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:29.328307 29076 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:29.330581 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:29.331194 29082 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:29.267510 31449 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:29.302927 31495 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:29.262089 31442 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.125778 31472 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:29.317067 31431 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:29.323465 31356 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:29.359958 29158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
W20260709 13:11:29.361668 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44597.8us
I20260709 13:11:29.364065 29081 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:29.364470 29158 raft_consensus.cc:606] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:29.364509 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:29.362512 31439 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:29.326403 31210 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:29.368453 28907 catalog_manager.cc:5697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 3, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:29.374984 31514 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:11:29.375655 31505 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:11:29.376031 31464 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:29.375301 31445 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:11:29.376680 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:29.376899 31508 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:11:29.378647 31513 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:11:29.380154 31519 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:11:29.381950 31518 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:11:29.386718 29076 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:29.389737 31461 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:29.266853 31451 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.391983 29079 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:29.390079 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:29.363662 31287 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:29.395712 31341 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:29.346451 31350 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:29.403609 31442 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:29.404198 31512 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:29.304791 31437 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:29.416918 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44209.1us
I20260709 13:11:29.420390 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:29.421790 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:29.423060 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:29.323923 31302 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.425060 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:29.387444 31352 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:29.427162 28907 catalog_manager.cc:5697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 3, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.330312 31448 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.440536 29048 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:29.442123 28907 catalog_manager.cc:5697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 4, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 4 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.444854 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:29.415711 31439 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.426087 31210 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:29.428012 31341 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:29.321959 31515 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.447435 28907 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 3 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:29.425161 31287 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:29.449198 31495 raft_consensus.cc:993] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:11:29.433530 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 397) took 1013 ms. Trace:
I20260709 13:11:29.446489 31445 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:29.439524 31351 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:29.446190 31442 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:29.462397 29076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:29.468629 29076 raft_consensus.cc:606] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:29.469244 29076 raft_consensus.cc:614] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:11:29.471544 29714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49414 (request call id 391) took 1899 ms (client timeout 1863 ms). Trace:
I20260709 13:11:29.475310 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:29.472293 29714 rpcz_store.cc:269] 0709 13:11:27.572619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:27.593182 (+ 20563us) service_pool.cc:224] Handling call
0709 13:11:29.471302 (+1878120us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400502154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:29.471450 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3174597888}
I20260709 13:11:29.476683 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:29.482492 28976 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:29.427841 31512 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.454245 31434 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:29.455996 31510 raft_consensus.cc:1081] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:11:29.455791 31341 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:29.502719 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:29.450773 31439 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:29.504254 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 467502us
W20260709 13:11:29.508504 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 195517us
W20260709 13:11:29.507669 31504 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.453s	user 0.001s	sys 0.000s
I20260709 13:11:29.501623 31433 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:29.507405 31348 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:29.511293 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:29.521991 29085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:29.487690 31516 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:11:29.503772 31421 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:29.487425 31464 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:29.503515 31368 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:11:29.536988 31464 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:29.537258 31448 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:29.537513 31464 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:29.526217 31433 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.541041 31433 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:29.510519 31169 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:11:29.533284 31260 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:29.512986 31351 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:29.454203 29708 rpcz_store.cc:276] 0709 13:11:28.420390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.424071 (+  3681us) service_pool.cc:224] Handling call
0709 13:11:28.424459 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:28.424839 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:28.425127 (+   288us) log.cc:844] Serialized 14 byte log entry
0709 13:11:28.680589 (+255462us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:28.681189 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 13:11:28.681363 (+   174us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:28.681416 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:28.681501 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:29.413639 (+732138us) raft_consensus.cc:1685] finished
0709 13:11:29.416154 (+  2515us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:29.427355 (+ 11201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":793,"prepare.queue_time_us":652,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":25958,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":27706,"thread_start_us":255057,"threads_started":1,"wal-append.queue_time_us":455298}
I20260709 13:11:29.555716 31512 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:29.512413 31434 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:29.556566 31527 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
W20260709 13:11:29.557435 31532 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:11:29.560876 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 82479us
I20260709 13:11:29.573820 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:29.575537 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:29.577016 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:29.565609 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54402.8us
I20260709 13:11:29.528946 31449 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:11:29.566838 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 122072us
I20260709 13:11:29.578526 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:29.579238 31449 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.579861 31449 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:29.588594 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:29.591667 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46226.7us
I20260709 13:11:29.594295 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:29.595146 29716 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:29.601419 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:29.603245 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.560372 31367 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.612403 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:29.612789 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:29.546288 31348 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:11:29.686563 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:29.688194 29008 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:29.690269 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:29.691279 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 78565.5us
I20260709 13:11:29.604616 31442 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:29.692236 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:29.690342 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:29.696691 29001 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:29.699599 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.621062 29006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:11:29.701197 29006 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.701602 29006 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:29.616384 31510 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:29.686820 31495 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:29.750579 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 58438.5us
I20260709 13:11:29.751636 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:29.685626 31260 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:29.687947 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 178719us
W20260709 13:11:29.712865 31511 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:11:29.617638 31356 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:29.705494 31266 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:29.765548 31529 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:29.711804 31451 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.687249 31454 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:29.789525 29715 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:29.760568 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:29.760322 31495 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.850980 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1235) took 1252 ms. Trace:
I20260709 13:11:29.854192 29001 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:29.861438 31533 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.760825 31249 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:11:29.863073 31543 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:11:29.864112 31526 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:29.619328 31367 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.750864 29710 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:11:29.943218 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 183321us
W20260709 13:11:29.942663 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 187568us
I20260709 13:11:29.943526 29156 rpcz_store.cc:276] 0709 13:11:28.598571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.658752 (+ 60181us) service_pool.cc:224] Handling call
0709 13:11:28.659195 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:28.659597 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:28.659635 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:28.671425 (+ 11790us) log.cc:844] Serialized 26 byte log entry
0709 13:11:28.873144 (+201719us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:28.873181 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:28.873249 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:29.850763 (+977514us) raft_consensus.cc:1685] finished
0709 13:11:29.850783 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:29.850919 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1198,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3682,"spinlock_wait_cycles":48256,"thread_start_us":125422,"threads_started":1,"wal-append.queue_time_us":477293}
I20260709 13:11:29.704579 31442 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:29.954538 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:29.955762 31496 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:29.957700 31496 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:29.959105 31442 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:29.938606 29006 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:29.766302 31351 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:29.960355 31496 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:29.964178 31496 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:29.965576 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:29.966804 31510 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:29.967471 31510 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:29.967803 29006 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:29.994824 29085 raft_consensus.cc:606] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:29.767019 31516 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.180s
I20260709 13:11:29.968531 29125 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:29.999131 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:29.995898 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:30.004199 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.004455 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.004983 29079 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:29.946296 31515 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:29.942078 31266 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:30.019984 29717 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:29.972083 31348 raft_consensus.cc:2764] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:11:30.023825 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:30.023849 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 444930us
I20260709 13:11:30.026397 29080 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:30.028679 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:29.997800 31523 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.006726 31535 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:30.018343 31529 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:29.931238 31451 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.019618 31445 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.035768 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:30.038172 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:30.010603 31454 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:11:30.054316 29715 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:29.968952 31533 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.040707 31434 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:30.011255 31449 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.043385 31523 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.046172 31529 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:30.144290 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1545) took 1717 ms. Trace:
I20260709 13:11:30.144522 29680 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:30.144739 29078 rpcz_store.cc:276] 0709 13:11:28.426717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.452340 (+ 25623us) service_pool.cc:224] Handling call
0709 13:11:28.457021 (+  4681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:28.632991 (+175970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.143850 (+1510859us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54001ad1d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:30.144026 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377716992}
I20260709 13:11:30.034821 31526 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:11:30.098312 29080 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:30.125099 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:30.080489 31481 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:30.082582 31544 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.147363 29715 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:30.083178 31540 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:30.090770 31439 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.173604 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.147650 31529 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:30.178925 31525 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.152s
I20260709 13:11:30.149133 31481 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:30.149454 31400 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:30.173031 31544 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.082947 31524 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.158s
I20260709 13:11:30.175441 31442 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.055248 31445 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:30.196966 29077 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.007s	sys 0.004s
W20260709 13:11:30.197427 29077 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.007s	sys 0.004s
I20260709 13:11:30.181079 31540 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:30.203442 31433 consensus_meta.cc:287] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.662s	user 0.004s	sys 0.005s
I20260709 13:11:30.170317 31448 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:11:30.184242 31356 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:30.210268 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 191066us
I20260709 13:11:30.190656 31400 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.212942 29006 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:11:30.216838 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.217612 29080 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:30.218165 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:30.218662 29001 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:30.220654 29009 raft_consensus.cc:2368] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:30.221396 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:30.222090 29009 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:30.092978 31510 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.226524 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:30.227293 29009 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:30.211669 31454 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.229655 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:30.223385 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.232973 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.230823 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.233717 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:30.231415 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:30.233726 29080 raft_consensus.cc:2410] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:30.234690 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:30.216269 31400 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:30.222657 31549 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:11:30.215631 31495 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:30.208639 31442 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.237524 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:30.242933 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:30.234071 29078 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:30.266950 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:30.204032 31433 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.213552 31535 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:11:30.279443 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:30.281286 31453 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:30.281798 31260 consensus_meta.cc:287] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.009s
I20260709 13:11:30.282207 29048 leader_election.cc:400] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:30.292343 29048 leader_election.cc:403] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:11:30.213941 31351 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:11:30.285290 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:30.294605 29080 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:30.282589 31509 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:11:30.285990 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.171600 31266 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.288637 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:30.299031 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:30.299970 29078 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:30.270942 31547 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:30.306556 31559 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:30.313939 31559 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:30.314754 31559 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.307175 29084 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:30.319643 31341 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.320243 31351 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:30.320897 31351 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:30.322119 31554 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:11:30.322954 31351 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:30.323470 31351 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:30.324585 31449 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.325955 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:30.326622 29154 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
W20260709 13:11:30.326894 31555 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:11:30.329676 29157 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:30.287420 31356 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:30.320099 31341 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.339020 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:30.312476 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1126) took 1854 ms. Trace:
I20260709 13:11:30.326304 29083 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:30.339931 29077 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:30.332248 29078 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:30.329404 29082 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:11:30.339797 29081 rpcz_store.cc:276] 0709 13:11:28.457951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.617362 (+159411us) service_pool.cc:224] Handling call
0709 13:11:29.363776 (+746414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:29.364498 (+   722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.312269 (+947771us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b5400045bd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:30.312411 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661918464}
I20260709 13:11:30.337312 29080 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
W20260709 13:11:30.311465 31557 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:11:30.292579 31495 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:30.296908 31521 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:30.305840 31464 consensus_meta.cc:287] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.768s	user 0.007s	sys 0.005s
I20260709 13:11:30.263063 31454 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.367985 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:30.369213 29157 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
W20260709 13:11:30.349412 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 67319.3us
I20260709 13:11:30.350958 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:30.351281 29078 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:30.373785 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:30.351333 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:30.320120 31552 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:11:30.375243 28976 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:30.375552 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:30.389366 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:30.393110 31434 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:30.393069 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:30.393631 31451 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.396643 29083 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:30.229251 31481 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.329371 31433 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.346179 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:30.347019 31448 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:30.323891 31351 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.429353 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:30.418948 29154 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:30.364795 31521 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.346596 31509 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:30.363808 31356 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:30.302874 31558 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.107s
I20260709 13:11:30.443437 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:30.443991 29077 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:30.407310 31025 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:30.447469 31434 raft_consensus.cc:697] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:30.448979 31434 consensus_queue.cc:237] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.427282 31525 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.427090 31524 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.426340 31455 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.476972 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.480600 29081 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:30.367727 31495 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.500882 31385 raft_consensus.cc:3055] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:30.501372 31385 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:30.501672 31495 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:30.502355 31385 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.504768 31529 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.395243 31464 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.506513 29713 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:30.430373 31533 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:30.503643 31385 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:30.481779 31453 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:30.486429 31455 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.487119 31524 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.527275 31524 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:30.488788 31556 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:11:30.527485 31455 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:30.489331 31351 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:30.491436 31571 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:30.437417 31521 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:30.494028 31544 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.496225 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:30.485708 31481 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:30.517536 29125 connection.cc:256] RPC call timeout handler was delayed by -1.1238544269999693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
W20260709 13:11:30.540831 29125 connection.cc:256] RPC call timeout handler was delayed by -1.0293544269999302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
I20260709 13:11:30.401686 31435 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:11:30.432807 31523 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:30.574244 29154 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:30.576052 29081 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:30.476140 31566 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:30.572161 29714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 574) took 1021 ms. Trace:
W20260709 13:11:30.570098 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 152252us
I20260709 13:11:30.588711 29714 rpcz_store.cc:276] 0709 13:11:29.550510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.550677 (+   167us) service_pool.cc:224] Handling call
0709 13:11:29.551112 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:29.590374 (+ 39262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:29.590403 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:29.606566 (+ 16163us) log.cc:844] Serialized 26 byte log entry
0709 13:11:29.930927 (+324361us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:29.930989 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:29.931095 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:30.571850 (+640755us) raft_consensus.cc:1685] finished
0709 13:11:30.571928 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:30.572102 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12440,"raft.queue_time_us":50097,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5296,"thread_start_us":238605,"threads_started":1,"wal-append.queue_time_us":465464}
I20260709 13:11:30.584831 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.584957 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:30.507366 31464 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:30.585608 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:30.591885 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42002 (request call id 19) took 1070 ms. Trace:
I20260709 13:11:30.483613 31368 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:30.593930 29085 rpcz_store.cc:276] 0709 13:11:29.521699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.521826 (+   127us) service_pool.cc:224] Handling call
0709 13:11:30.591734 (+1069908us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400023d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583b853 00007f411a80960a 00007f4113d8f7d4 00007f4113d8f762 00007f4113d8f691 00007f4113d8f60c 00007f4113d8e8c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:30.591826 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070581504}
I20260709 13:11:30.597469 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.609476 31368 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.609988 31368 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:30.433296 31448 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.620896 29085 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:30.621912 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:30.301017 31540 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.566879 31025 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:30.648698 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:30.651412 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:30.652256 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:30.659441 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:30.659482 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1549) took 2225 ms. Trace:
I20260709 13:11:30.660142 29712 raft_consensus.cc:2376] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6. Re-sending same reply.
I20260709 13:11:30.660897 29712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 364) took 1084 ms. Trace:
I20260709 13:11:30.660228 29084 rpcz_store.cc:276] 0709 13:11:28.427857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.428141 (+   284us) service_pool.cc:224] Handling call
0709 13:11:28.428399 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:30.319303 (+1890904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.653158 (+333855us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b540027b0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:30.653380 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4312227584}
I20260709 13:11:30.661392 29712 rpcz_store.cc:276] 0709 13:11:29.576566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.608875 (+ 32309us) service_pool.cc:224] Handling call
0709 13:11:30.659288 (+1050413us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:30.660828 (+  1540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4325120}
I20260709 13:11:30.624292 31445 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:30.667040 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 88888.1us
W20260709 13:11:30.663573 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 218972us
I20260709 13:11:30.670442 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:30.512408 31525 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.622596 31400 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:30.698403 31517 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.376s	user 0.005s	sys 0.007s
I20260709 13:11:30.684494 31533 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:30.702011 31525 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:30.663990 31535 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:30.704715 31535 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.708567 31580 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:30.709169 31580 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.668097 31442 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.689278 29682 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:30.664674 31356 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.856916 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 360) took 1213 ms. Trace:
I20260709 13:11:30.771786 29157 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:30.730099 29155 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:11:30.776013 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:30.907482 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 218250us
I20260709 13:11:30.908809 29155 pending_rounds.cc:85] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 5
I20260709 13:11:30.915747 29155 pending_rounds.cc:107] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:11:30.916435 29155 raft_consensus.cc:2889] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:30.892967 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:30.921607 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1232) took 2517 ms. Trace:
I20260709 13:11:30.712602 31464 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:30.913035 31591 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:11:30.906196 29713 rpcz_store.cc:276] 0709 13:11:29.575063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.612299 (+ 37236us) service_pool.cc:224] Handling call
0709 13:11:29.690272 (+ 77973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:30.506850 (+816578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.784804 (+277954us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b540021f554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:30.788755 (+  3951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1876470144}
I20260709 13:11:30.897611 31449 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:30.902138 31580 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.867015 31454 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.921981 29150 rpcz_store.cc:276] 0709 13:11:28.390325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.390621 (+   296us) service_pool.cc:224] Handling call
0709 13:11:28.390873 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:28.405687 (+ 14814us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:28.596284 (+190597us) log.cc:844] Serialized 14 byte log entry
0709 13:11:28.700199 (+103915us) log.cc:844] Serialized 14 byte log entry
0709 13:11:28.700415 (+   216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:28.700473 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:28.700527 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.907563 (+2207036us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b54003edfd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:30.907613 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:30.907742 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23544,"prepare.queue_time_us":210549,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":20004,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":208,"spinlock_wait_cycles":33606016,"thread_start_us":266619,"threads_started":2,"wal-append.queue_time_us":343782,"wal-append.run_cpu_time_us":2889,"wal-append.run_wall_time_us":1406172}
I20260709 13:11:30.912803 31448 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:30.914103 31312 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:30.911746 31302 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.890605 31535 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:30.962158 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:30.935411 31451 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.926437 31455 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:30.715464 31515 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.214s
I20260709 13:11:30.788545 31368 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.985947 31368 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.965667 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:30.986864 29158 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:30.976302 31451 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:30.977492 31400 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:30.956861 31449 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:30.978667 31545 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.030s	user 0.002s	sys 0.000s
I20260709 13:11:30.979259 29717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 356) took 1407 ms. Trace:
W20260709 13:11:30.980333 31586 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:11:30.936506 31169 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:11:31.012269 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2007) took 2425 ms. Trace:
I20260709 13:11:31.012241 29717 rpcz_store.cc:276] 0709 13:11:29.571924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.572098 (+   174us) service_pool.cc:224] Handling call
0709 13:11:29.572473 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:30.020558 (+448085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.979006 (+958448us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54006b0dd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:30.979192 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004569984}
I20260709 13:11:31.015199 29159 rpcz_store.cc:276] 0709 13:11:28.586654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.590529 (+  3875us) service_pool.cc:224] Handling call
0709 13:11:28.590861 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:28.665859 (+ 74998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:31.011987 (+2346128us) spinlock_profiling.cc:243] Waited 72 ms on lock 0x7b5400111f54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:31.012212 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165712128}
I20260709 13:11:30.757843 31025 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:30.940116 31464 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.013388 31529 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:31.040158 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:31.053658 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:31.054549 29080 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:31.055574 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:31.067205 31481 consensus_meta.cc:287] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
I20260709 13:11:31.068054 31451 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:31.073557 31340 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:31.074980 31340 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.073783 31576 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:31.076742 31449 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:31.076764 31576 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.079101 31464 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:31.056533 31385 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.002s
I20260709 13:11:31.086305 31340 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:31.065706 31432 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.088618 31432 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.091360 31432 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:31.079736 31576 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:31.096127 29080 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:31.097409 29080 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:31.099558 31569 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:31.100111 31569 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:31.101095 31569 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.102329 31569 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:31.098340 29080 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.103281 29007 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:31.103494 29080 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:31.120739 31352 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:31.123207 28975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.183s	user 0.012s	sys 0.006s
W20260709 13:11:31.123625 28975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.183s	user 0.012s	sys 0.006s
I20260709 13:11:31.127592 31367 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:31.131994 31521 consensus_meta.cc:287] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.004s
W20260709 13:11:31.134652 31567 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.692s	user 0.007s	sys 0.000s
I20260709 13:11:31.135296 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:31.136247 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 316) took 1024 ms. Trace:
I20260709 13:11:31.136631 29076 rpcz_store.cc:276] 0709 13:11:30.111663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.126731 (+ 15068us) service_pool.cc:224] Handling call
0709 13:11:30.127015 (+   284us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:30.127333 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:30.127351 (+    18us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:30.148352 (+ 21001us) log.cc:844] Serialized 52 byte log entry
0709 13:11:30.283018 (+134666us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:30.283070 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.283139 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:31.118766 (+835627us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:31.135894 (+ 17128us) raft_consensus.cc:1685] finished
0709 13:11:31.135913 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:31.136182 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6766,"spinlock_wait_cycles":76176256,"thread_start_us":99342,"threads_started":1,"wal-append.queue_time_us":180333}
I20260709 13:11:31.138799 29076 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:31.139107 29076 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:31.139807 29076 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.140823 29076 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:31.143159 30921 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:31.145597 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1630) took 2759 ms. Trace:
I20260709 13:11:31.140848 31348 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:11:31.146086 29010 rpcz_store.cc:276] 0709 13:11:28.386456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.386571 (+   115us) service_pool.cc:224] Handling call
0709 13:11:28.405955 (+ 19384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:28.891460 (+485505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:31.145310 (+2253850us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b5400228654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:31.145524 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103057280}
W20260709 13:11:31.148788 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 2.26069e+06us
I20260709 13:11:31.152669 29711 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:31.153873 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47384 (request call id 578) took 1592 ms. Trace:
I20260709 13:11:31.137146 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:31.161641 29076 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:11:31.163901 29076 pending_rounds.cc:85] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 2
I20260709 13:11:31.165836 29076 pending_rounds.cc:107] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:11:31.167712 29076 raft_consensus.cc:2889] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:31.171104 31448 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.171900 31448 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.173735 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:31.178957 29076 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:31.182163 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:31.174464 31448 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:31.187286 31506 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.189081 29083 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:31.154264 29711 rpcz_store.cc:276] 0709 13:11:29.561754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.589934 (+ 28180us) service_pool.cc:224] Handling call
0709 13:11:31.135130 (+1545196us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ceee 0000000000375939 00007f411a7901e0 00007f411a8590c4 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:31.153801 (+ 18671us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3002112}
I20260709 13:11:31.210464 31431 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:31.175079 31385 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:31.171223 31428 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:31.192099 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:31.187834 31506 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.216082 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.217895 29083 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:31.188027 31576 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.219861 31421 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.220392 31350 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:31.213780 31566 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:31.224681 31341 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:31.228474 31506 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:31.230688 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.326071 29009 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:31.232816 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:31.327536 29083 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:31.132592 31521 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:31.238899 31260 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 13:11:31.249475 31568 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.812s	user 0.003s	sys 0.001s
I20260709 13:11:31.211244 31431 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.341039 31431 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:31.282287 29080 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:11:31.335704 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.341737 29080 pending_rounds.cc:85] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 2
I20260709 13:11:31.341944 29005 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 3.
I20260709 13:11:31.337656 31575 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:31.341985 29080 pending_rounds.cc:107] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:11:31.344211 29080 raft_consensus.cc:2889] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:31.344451 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.341389 31421 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.346853 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:31.345729 29051 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:31.347654 29077 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:31.352463 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:31.353950 31559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.023s	sys 0.036s
W20260709 13:11:31.358421 31559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.023s	sys 0.036s
W20260709 13:11:31.359572 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 199950us
I20260709 13:11:31.359901 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:31.361368 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:31.362968 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:31.363620 29080 raft_consensus.cc:2376] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5. Re-sending same reply.
I20260709 13:11:31.364950 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:31.365438 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:31.366358 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:31.356177 29083 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:31.363620 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:31.368963 29083 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:11:31.366228 29077 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:31.371506 29080 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:31.375247 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.376850 29001 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:11:31.350108 31441 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:11:31.379165 29005 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:31.379644 29006 raft_consensus.cc:2410] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:11:31.379927 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:31.386991 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.360419 31400 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.391278 31400 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.393965 31400 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:31.396225 31526 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.396865 31526 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.399097 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:31.399413 31526 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:31.399951 31495 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:31.401619 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:31.402540 29155 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:31.403650 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:31.405217 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:31.405925 29156 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:31.401647 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:31.407022 31472 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:31.407374 31472 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:31.407416 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:31.407868 31472 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:31.408525 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:31.408370 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:31.409294 29153 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:11:31.410975 31139 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:31.411388 31139 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:31.411737 31139 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:31.412216 31461 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:31.412665 31461 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:31.413100 31461 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:31.418015 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:31.412453 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:31.387795 29001 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:31.420332 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.422838 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:31.436187 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:31.437219 29155 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:31.440307 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:31.445215 29050 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:31.355189 31448 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:31.324678 31576 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:31.456919 28975 connection.cc:256] RPC call timeout handler was delayed by -1.5641647340000873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:11:31.483484 28975 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:31.485499 28975 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:31.458968 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:31.460777 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.487361 29007 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
W20260709 13:11:31.419129 29048 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.976s	user 0.032s	sys 0.079s
W20260709 13:11:31.488353 29048 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.032s	sys 0.079s
I20260709 13:11:31.342866 31575 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.501505 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:31.464839 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:31.416376 31351 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.440784 31495 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.466769 31542 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:31.468138 31260 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:31.470024 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.330765 31350 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.444162 31526 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:31.489152 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 1.09974e+06us
I20260709 13:11:31.534909 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:31.517092 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:31.385535 31385 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:31.385780 31438 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:31.391305 31266 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:31.470050 31506 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:31.531606 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 72746.2us
I20260709 13:11:31.537912 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:31.457947 31509 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:31.542364 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:31.546533 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:31.547816 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:31.469192 31448 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:31.576339 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:31.509032 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:31.594568 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:31.562556 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:31.564003 31421 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:31.566223 31454 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:31.549741 31340 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:31.531356 31437 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:31.525482 31495 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.573145 31266 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.601186 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:31.605856 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 49474.1us
I20260709 13:11:31.572021 31576 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:31.629396 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:31.634541 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 192752us
W20260709 13:11:31.634903 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 256354us
I20260709 13:11:31.551295 31481 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:31.638968 29007 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:11:31.641635 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:31.644222 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:31.643796 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:31.662292 29159 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:31.662673 29159 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:31.644876 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:31.604457 31509 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.595075 31448 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:31.701025 29157 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:31.631299 31580 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.654496 29004 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:31.647430 29005 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
W20260709 13:11:31.599215 31510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.015s	sys 0.061s
I20260709 13:11:31.668210 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:31.668133 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:31.594305 31588 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:31.671317 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:31.532553 31351 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.669968 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:31.705304 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 213656us
I20260709 13:11:31.634277 31421 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:31.733773 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 62533.2us
I20260709 13:11:31.734637 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:31.634619 31512 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:31.555512 31438 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:31.718861 31510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.016s	sys 0.061s
W20260709 13:11:31.723713 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53899.2us
I20260709 13:11:31.713521 29156 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
W20260709 13:11:31.724764 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 65739.2us
W20260709 13:11:31.669413 31004 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.162s	user 0.023s	sys 0.100s
I20260709 13:11:31.574105 31385 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.595433 31506 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:31.720213 29084 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.007s	sys 0.031s
I20260709 13:11:31.636104 31464 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:11:31.669178 31490 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:31.645361 31348 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.087s
I20260709 13:11:31.632787 31454 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.647800 31437 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:31.655224 31524 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 13:11:31.834651 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:31.715164 29153 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:31.878093 29155 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
W20260709 13:11:31.845459 29082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.009s	sys 0.027s
I20260709 13:11:31.820812 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:31.845952 29154 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.012s	sys 0.009s
W20260709 13:11:31.880257 29154 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.987s	user 0.013s	sys 0.010s
I20260709 13:11:31.714421 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:31.744907 31533 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:31.856282 29084 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.126s	user 0.007s	sys 0.031s
I20260709 13:11:31.737834 31472 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.658332 31491 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.153s
I20260709 13:11:31.744632 31529 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:31.863796 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:31.910730 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:31.916535 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52847.4us
W20260709 13:11:31.709352 31434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.216s	user 0.037s	sys 0.100s
W20260709 13:11:31.834065 31004 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.319s	user 0.023s	sys 0.101s
I20260709 13:11:31.701066 31249 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.136s
I20260709 13:11:31.820772 31526 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:31.869830 29712 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:11:31.876992 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:31.718132 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:31.850461 31452 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.852731 31579 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.721153 31580 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.718341 31481 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:31.904860 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 84141.4us
W20260709 13:11:31.967057 29078 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.037s	sys 0.058s
W20260709 13:11:31.968408 29078 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.037s	sys 0.058s
I20260709 13:11:31.883694 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:31.961503 29150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.003s	sys 0.011s
W20260709 13:11:31.880856 29082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.010s	sys 0.027s
I20260709 13:11:31.746917 31421 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.903443 31368 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
W20260709 13:11:31.873994 31562 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.893s	user 0.033s	sys 0.020s
I20260709 13:11:31.718202 31509 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:31.869401 31340 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:31.955946 29081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.009s	sys 0.004s
W20260709 13:11:32.015267 29081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.009s	sys 0.004s
I20260709 13:11:31.867386 31583 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:31.876164 31490 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:31.876863 31575 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:31.963138 31526 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:31.915027 31533 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:31.912809 31521 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:32.022878 31433 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.009s	sys 0.055s
I20260709 13:11:31.914737 31542 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:32.060523 31495 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:32.060899 31597 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:11:32.061628 31454 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:31.913554 31445 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:32.064323 31445 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:32.048767 31526 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:32.066473 31445 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:31.916095 31529 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:32.067066 31529 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:32.051744 31533 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:31.960398 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 83485.2us
W20260709 13:11:31.969548 29150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.003s	sys 0.011s
I20260709 13:11:32.068892 29680 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:32.069172 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2021) took 1163 ms. Trace:
I20260709 13:11:32.069592 29150 rpcz_store.cc:276] 0709 13:11:30.906269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.968492 (+ 62223us) service_pool.cc:224] Handling call
0709 13:11:30.970071 (+  1579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:30.970325 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:31.016431 (+ 46106us) log.cc:844] Serialized 14 byte log entry
0709 13:11:32.068848 (+1052417us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:32.068900 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:32.068943 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.069006 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:32.069119 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":43012,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":39,"thread_start_us":1052034,"threads_started":1}
W20260709 13:11:31.913455 29003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.017s	sys 0.021s
I20260709 13:11:32.071224 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:32.071477 29003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.018s	sys 0.021s
W20260709 13:11:31.973049 29076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.008s	sys 0.007s
I20260709 13:11:32.072194 31464 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:32.072464 29076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.008s	sys 0.007s
I20260709 13:11:32.072629 31464 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:32.072465 31472 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:32.073123 31464 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:32.073766 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:32.074317 29150 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:31.980985 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:32.075062 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 94560.2us
I20260709 13:11:31.953431 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:32.076022 29153 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:31.903878 31455 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 13:11:32.007210 29152 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.236s	user 0.015s	sys 0.054s
I20260709 13:11:32.076529 31553 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:32.076870 29152 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.319s	user 0.015s	sys 0.054s
I20260709 13:11:32.077047 31553 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:32.077543 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2020) took 1362 ms. Trace:
W20260709 13:11:32.078390 31569 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.976s	user 0.007s	sys 0.004s
W20260709 13:11:32.078557 31600 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:11:32.079772 31490 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:32.078598 31496 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:32.081359 31496 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:32.081717 31496 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:32.082432 31351 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:32.083009 31351 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:11:32.057904 31593 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:11:31.716529 31480 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.165s
I20260709 13:11:32.086143 31509 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:32.087159 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1567) took 1539 ms. Trace:
I20260709 13:11:31.965325 31450 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:31.966254 31435 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:32.088219 31435 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:31.945780 31434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.453s	user 0.038s	sys 0.100s
I20260709 13:11:32.009912 31249 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:11:31.981307 31524 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:31.968147 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:32.007247 31562 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.034s	sys 0.020s
I20260709 13:11:31.979651 31266 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:31.981840 29158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.038s	sys 0.051s
I20260709 13:11:32.008029 31350 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:31.710757 29080 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
W20260709 13:11:31.959025 31540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.067s	user 0.014s	sys 0.028s
W20260709 13:11:32.056635 31599 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:11:32.091401 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:32.057371 31598 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:11:32.057857 31421 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:32.059185 31437 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:32.064219 31481 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:32.084262 31506 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:31.980239 31452 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:32.135485 31441 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:32.136006 31441 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:32.087592 31438 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:32.027120 31583 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:32.137573 31461 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.724s	user 0.009s	sys 0.000s
I20260709 13:11:32.137967 31441 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:32.138556 31441 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:32.139180 31583 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:32.142627 31583 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:32.143090 31583 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:32.143611 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:32.006158 31341 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.005s	sys 0.014s
W20260709 13:11:32.144234 31341 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.005s	sys 0.014s
I20260709 13:11:32.144398 29076 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:32.088630 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:32.145090 29076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "243d4ae4a4ae48b2979e32874472d256"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:32.145617 29076 raft_consensus.cc:606] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Received request to transfer leadership to TS 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:32.145908 29076 raft_consensus.cc:614] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:32.146265 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:32.146808 29153 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:32.147455 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:32.090632 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:32.088624 31435 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:32.148540 29080 raft_consensus.cc:2410] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:32.089354 29150 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
W20260709 13:11:32.091115 29158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.039s	sys 0.056s
I20260709 13:11:32.087539 29079 rpcz_store.cc:276] 0709 13:11:30.547045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.547232 (+   187us) service_pool.cc:224] Handling call
0709 13:11:30.547626 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:30.548087 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:30.558756 (+ 10669us) log.cc:844] Serialized 14 byte log entry
0709 13:11:31.009878 (+451122us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.011678 (+  1800us) log.cc:844] Serialized 26 byte log entry
0709 13:11:31.011830 (+   152us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:31.015003 (+  3173us) log.cc:844] Serialized 14 byte log entry
0709 13:11:31.017924 (+  2921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:31.017998 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:31.149317 (+131319us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b1800436480. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:11:31.159851 (+ 10534us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b1800436480. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:11:32.086535 (+926684us) raft_consensus.cc:1685] finished
0709 13:11:32.086557 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:32.086695 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2661,"prepare.queue_time_us":210154,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":11622,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":1636,"spinlock_wait_cycles":6723584,"thread_start_us":460382,"threads_started":2,"wal-append.queue_time_us":1072665}
I20260709 13:11:32.026219 31512 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:32.089172 31524 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:32.152691 29079 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:11:32.055189 31433 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.009s	sys 0.055s
I20260709 13:11:32.154162 31352 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:32.154368 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:32.154973 29150 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:32.154699 31352 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:32.155737 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:32.156261 29150 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:32.107293 31266 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:31.980087 31579 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:32.114063 31139 consensus_meta.cc:287] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.661s	user 0.006s	sys 0.003s
I20260709 13:11:32.111730 31464 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:32.129410 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:32.139029 31441 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:32.022668 31340 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:32.092175 31540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.200s	user 0.014s	sys 0.029s
I20260709 13:11:32.151579 31512 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:32.143774 29159 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:32.147897 29153 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:32.447355 29713 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.019s	sys 0.016s
W20260709 13:11:32.677287 29713 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.020s	sys 0.025s
W20260709 13:11:32.677400 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:32.385833 29716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.010s	sys 0.023s
W20260709 13:11:32.679666 29716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.018s	sys 0.027s
W20260709 13:11:32.379515 29151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.016s	sys 0.035s
W20260709 13:11:32.680516 29151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.266s	user 0.023s	sys 0.039s
W20260709 13:11:32.266533 29083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.006s	sys 0.023s
W20260709 13:11:32.681558 29083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.006s	sys 0.040s
I20260709 13:11:32.566902 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 303) took 1171 ms. Trace:
I20260709 13:11:32.077890 29152 rpcz_store.cc:276] 0709 13:11:30.714847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.715331 (+   484us) service_pool.cc:224] Handling call
0709 13:11:30.715750 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:30.716010 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:30.757388 (+ 41378us) log.cc:844] Serialized 14 byte log entry
0709 13:11:32.077231 (+1319843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:32.077277 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:32.077331 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.077391 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:32.077492 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110543,"prepare.queue_time_us":55550,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":7091,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":1319543,"threads_started":1}
I20260709 13:11:32.683096 29159 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:32.683235 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:32.288424 29085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.301s	user 0.024s	sys 0.039s
W20260709 13:11:32.684538 29085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.697s	user 0.033s	sys 0.046s
I20260709 13:11:32.684713 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:32.689206 29077 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.084s	user 0.012s	sys 0.016s
W20260709 13:11:32.690064 29077 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.085s	user 0.013s	sys 0.016s
W20260709 13:11:32.692777 29051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.007s	sys 0.019s
W20260709 13:11:32.694607 29051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.007s	sys 0.020s
W20260709 13:11:32.692131 29048 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.015s	sys 0.011s
W20260709 13:11:32.698058 29048 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.016s	sys 0.011s
W20260709 13:11:32.678925 29150 consensus_meta.cc:287] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.522s	user 0.016s	sys 0.001s
I20260709 13:11:32.700651 29150 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:32.682453 29158 rpcz_store.cc:276] 0709 13:11:31.395595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.395728 (+   133us) service_pool.cc:224] Handling call
0709 13:11:31.396142 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:31.396485 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:31.396515 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.396997 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:11:32.150707 (+753710us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:32.150740 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.150824 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:32.566599 (+415775us) raft_consensus.cc:1685] finished
0709 13:11:32.566628 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:32.566816 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1692,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":753363,"threads_started":1,"wal-append.queue_time_us":1072134}
I20260709 13:11:32.702005 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1259) took 1042 ms. Trace:
I20260709 13:11:32.703940 29150 rpcz_store.cc:276] 0709 13:11:31.659771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.155531 (+495760us) service_pool.cc:224] Handling call
0709 13:11:32.701934 (+546403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:32.706738 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:32.702675 29154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.010s	sys 0.015s
W20260709 13:11:32.710155 29154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.010s	sys 0.015s
I20260709 13:11:32.711479 29150 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:32.714112 29155 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:11:32.716130 29122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.002s	sys 0.021s
W20260709 13:11:32.716635 29122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.003s	sys 0.022s
W20260709 13:11:32.717114 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 982446us
W20260709 13:11:32.718088 28974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.006s	sys 0.013s
W20260709 13:11:32.761230 28974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.006s	sys 0.013s
W20260709 13:11:32.678309 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:32.733139 29005 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.020s	sys 0.021s
W20260709 13:11:32.763520 29005 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.025s	sys 0.030s
W20260709 13:11:32.695060 29082 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.002s	sys 0.002s
W20260709 13:11:32.764390 29082 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.004s	sys 0.008s
W20260709 13:11:32.217619 29002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.011s	sys 0.011s
W20260709 13:11:32.765270 29002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.502s	user 0.014s	sys 0.019s
I20260709 13:11:32.776238 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:32.795954 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:32.598587 31616 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:11:32.584415 31533 consensus_meta.cc:287] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
W20260709 13:11:32.601171 31529 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
W20260709 13:11:32.217382 31446 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.009s	sys 0.013s
W20260709 13:11:32.616731 31448 consensus_meta.cc:287] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.883s	user 0.010s	sys 0.000s
W20260709 13:11:32.597227 31611 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.286s	user 0.001s	sys 0.003s
W20260709 13:11:32.691444 31432 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.007s	sys 0.007s
W20260709 13:11:32.691778 31431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.009s	sys 0.007s
W20260709 13:11:32.968245 31431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.011s	sys 0.007s
W20260709 13:11:32.277700 31356 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.028s	sys 0.065s
I20260709 13:11:33.005944 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:32.801151 29155 raft_consensus.cc:2393] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:32.332650 31025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.249s	user 0.005s	sys 0.018s
I20260709 13:11:32.804706 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1573) took 2219 ms. Trace:
I20260709 13:11:33.010244 29082 rpcz_store.cc:276] 0709 13:11:30.585091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.670923 (+ 85832us) service_pool.cc:224] Handling call
0709 13:11:30.693614 (+ 22691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:30.694045 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:30.694079 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:30.694560 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:11:31.969346 (+1274786us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:32.020476 (+ 51130us) log.cc:844] Serialized 14 byte log entry
0709 13:11:32.134242 (+113766us) spinlock_profiling.cc:243] Waited 895 us on lock 0x7b1800346900. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:11:32.764871 (+630629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.764952 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:32.765008 (+    56us) raft_consensus.cc:1685] finished
0709 13:11:32.804476 (+ 39468us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:32.804644 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48873,"prepare.queue_time_us":53104,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3867,"raft.queue_time_us":122012,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":1127,"spinlock_wait_cycles":2058112,"thread_start_us":1821704,"threads_started":2,"wal-append.queue_time_us":1085563}
I20260709 13:11:32.805780 29150 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:32.672349 31170 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:32.670492 31604 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.361s	user 0.001s	sys 0.002s
I20260709 13:11:32.309201 31249 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:32.692658 29708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.002s	sys 0.024s
I20260709 13:11:32.316164 31450 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:32.693239 31510 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:32.483032 31352 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:32.722136 31139 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:32.914309 29153 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:32.700322 31553 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:32.960944 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:32.602124 31579 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:32.721758 31583 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:32.092579 31569 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.734s
I20260709 13:11:32.184928 31452 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:32.964741 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1641) took 3776 ms. Trace:
W20260709 13:11:32.960630 31432 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.008s	sys 0.008s
I20260709 13:11:32.229946 31266 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:32.837354 31526 consensus_meta.cc:287] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.772s	user 0.003s	sys 0.008s
I20260709 13:11:32.802205 31442 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:32.962935 31446 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.546s	user 0.013s	sys 0.024s
I20260709 13:11:32.826207 31455 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:32.137991 31461 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:32.615077 31524 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:33.907233 31608 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:11:33.926343 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:33.926206 31266 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:33.904620 31464 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:33.561498 31621 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.407s	user 0.000s	sys 0.000s
I20260709 13:11:33.918169 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1569) took 3362 ms. Trace:
I20260709 13:11:33.928872 31642 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:33.562021 31615 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.968s	user 0.000s	sys 0.000s
I20260709 13:11:33.929328 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 329) took 2549 ms. Trace:
W20260709 13:11:33.600688 29078 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.575s	user 0.037s	sys 0.066s
W20260709 13:11:33.810752 31607 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.947s	user 0.001s	sys 0.001s
W20260709 13:11:33.585457 31385 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.199s	user 0.035s	sys 0.062s
W20260709 13:11:33.308832 31428 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.329s	user 0.012s	sys 0.036s
W20260709 13:11:33.937595 31428 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.056s	user 0.016s	sys 0.041s
I20260709 13:11:33.936167 29077 rpcz_store.cc:276] 0709 13:11:31.380458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.383944 (+  3486us) service_pool.cc:224] Handling call
0709 13:11:31.462289 (+ 78345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:31.493569 (+ 31280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:31.493616 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.494184 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:11:32.690977 (+1196793us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:32.691061 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.691238 (+   177us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:33.929125 (+1237887us) raft_consensus.cc:1685] finished
0709 13:11:33.929145 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:33.929274 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30961,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":16667,"thread_start_us":1085354,"threads_started":1,"wal-append.queue_time_us":1270135}
I20260709 13:11:33.938679 31579 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:33.939354 29708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.984s	user 0.021s	sys 0.032s
W20260709 13:11:33.654433 29680 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.583s	user 0.047s	sys 0.059s
W20260709 13:11:33.940562 29680 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.869s	user 0.049s	sys 0.065s
I20260709 13:11:32.841814 31533 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:33.940958 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 1.8696e+06us
I20260709 13:11:33.942126 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:33.942984 31533 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:33.233076 31578 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.124s	user 0.008s	sys 0.013s
W20260709 13:11:33.944432 31578 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.835s	user 0.011s	sys 0.019s
W20260709 13:11:33.814286 31490 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.727s	user 0.017s	sys 0.018s
I20260709 13:11:33.945331 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:33.945768 31490 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.863s	user 0.017s	sys 0.020s
I20260709 13:11:33.917966 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:33.947959 31648 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:33.948132 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:33.948659 31139 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:33.949150 31622 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:32.961930 31435 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.813s	user 0.005s	sys 0.006s
I20260709 13:11:33.949967 31139 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:33.950219 31435 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:33.951023 29711 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:33.951694 29159 consensus_meta.cc:287] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.191s	user 0.008s	sys 0.003s
I20260709 13:11:33.952086 29159 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:33.952526 31461 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:33.952620 29153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1272) took 1894 ms (client timeout 1773 ms). Trace:
W20260709 13:11:33.232558 29076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.075s	user 0.001s	sys 0.018s
W20260709 13:11:33.953253 29153 rpcz_store.cc:269] 0709 13:11:32.057718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.147264 (+ 89546us) service_pool.cc:224] Handling call
0709 13:11:33.952508 (+1805244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:33.943089 31464 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:11:33.580404 29084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.592s	user 0.024s	sys 0.076s
W20260709 13:11:33.955468 29084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.987s	user 0.025s	sys 0.088s
I20260709 13:11:33.928830 29081 rpcz_store.cc:276] 0709 13:11:30.555849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.650206 (+ 94357us) service_pool.cc:224] Handling call
0709 13:11:30.668197 (+ 17991us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:30.677098 (+  8901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:30.677146 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:30.686019 (+  8873us) log.cc:844] Serialized 26 byte log entry
0709 13:11:31.163969 (+477950us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b18003c3be0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:11:32.021048 (+857079us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:32.021083 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.021150 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:32.596527 (+575377us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:33.917930 (+1321403us) raft_consensus.cc:1685] finished
0709 13:11:33.917948 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:33.918100 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1910,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":14568,"spinlock_wait_cycles":72530688,"thread_start_us":856891,"threads_started":1,"wal-append.queue_time_us":1614865}
I20260709 13:11:33.956112 31583 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:33.956686 31583 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:33.937608 31249 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:33.959192 31583 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:33.959457 29152 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:33.960786 31650 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:33.006771 31496 consensus_meta.cc:287] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.925s	user 0.007s	sys 0.004s
I20260709 13:11:33.962591 31496 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:33.940582 31442 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:33.751428 31004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.747s	user 0.006s	sys 0.030s
W20260709 13:11:33.232950 28976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.315s	user 0.000s	sys 0.021s
W20260709 13:11:33.968528 28976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.051s	user 0.000s	sys 0.031s
W20260709 13:11:33.968899 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 2.05092e+06us
I20260709 13:11:32.961000 31588 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.970063 31588 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:33.928114 29002 rpcz_store.cc:276] 0709 13:11:29.188125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.188258 (+   133us) service_pool.cc:224] Handling call
0709 13:11:29.188504 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:29.188708 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:29.366080 (+177372us) log.cc:844] Serialized 14 byte log entry
0709 13:11:31.262770 (+1896690us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b1800366e20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287
0709 13:11:31.263151 (+   381us) log.cc:844] Serialized 14 byte log entry
0709 13:11:32.964291 (+1701140us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:32.964365 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:32.964429 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.964525 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:32.964686 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":259566,"prepare.queue_time_us":150783,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":99,"raft.queue_time_us":39750,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":17424,"spinlock_wait_cycles":9850112,"thread_start_us":1792064,"threads_started":4,"wal-append.queue_time_us":168540,"wal-append.run_cpu_time_us":1129,"wal-append.run_wall_time_us":1001147}
W20260709 13:11:33.232769 29715 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.408s	user 0.005s	sys 0.079s
W20260709 13:11:33.974633 29715 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.149s	user 0.005s	sys 0.088s
W20260709 13:11:33.952687 29159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1256) took 2361 ms (client timeout 1774 ms). Trace:
W20260709 13:11:33.919771 28975 connection.cc:256] RPC call timeout handler was delayed by -1.20273435099989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
W20260709 13:11:33.976302 28975 connection.cc:256] RPC call timeout handler was delayed by -1.1730343509999557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 13:11:33.976758 28975 connection.cc:256] RPC call timeout handler was delayed by -1.165834350999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
I20260709 13:11:33.793491 31340 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:33.977231 28975 connection.cc:256] RPC call timeout handler was delayed by -1.0803343509999195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
I20260709 13:11:33.969411 31647 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.262190 31628 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.001307 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:33.973445 29713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.295s	user 0.000s	sys 0.020s
W20260709 13:11:33.980064 29050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.217s	user 0.000s	sys 0.010s
W20260709 13:11:33.969029 29078 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.945s	user 0.042s	sys 0.073s
W20260709 13:11:33.980631 29717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.000s	sys 0.009s
W20260709 13:11:33.981161 29051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.221s	user 0.003s	sys 0.007s
W20260709 13:11:33.980916 29125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.219s	user 0.008s	sys 0.002s
I20260709 13:11:33.961499 31650 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:34.007342 29713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.329s	user 0.000s	sys 0.021s
W20260709 13:11:33.937139 31385 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.558s	user 0.044s	sys 0.066s
I20260709 13:11:33.326223 29082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:33.005568 29709 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:33.167464 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:33.986153 29085 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.301s	user 0.003s	sys 0.008s
I20260709 13:11:34.002300 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:34.005442 29050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.000s	sys 0.011s
W20260709 13:11:34.005558 29051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.245s	user 0.003s	sys 0.007s
W20260709 13:11:33.998266 31480 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 13:11:33.006386 31356 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.889s	user 0.032s	sys 0.078s
W20260709 13:11:33.975654 29159 rpcz_store.cc:269] 0709 13:11:31.591546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.600689 (+  9143us) service_pool.cc:224] Handling call
0709 13:11:32.143405 (+542716us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540042cd20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157bf06f 00007f4115837f60 00007f41158397de 00007f411584527c 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2
0709 13:11:33.952600 (+1809195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817478912}
W20260709 13:11:33.900547 31618 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
W20260709 13:11:33.902350 31605 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.349s	user 0.000s	sys 0.001s
I20260709 13:11:33.552353 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2037) took 1054 ms. Trace:
I20260709 13:11:34.002278 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2038) took 1174 ms. Trace:
I20260709 13:11:34.008575 29082 raft_consensus.cc:606] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:33.623835 29083 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.002s	sys 0.009s
I20260709 13:11:34.040695 29155 rpcz_store.cc:276] 0709 13:11:32.498044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.775838 (+277794us) service_pool.cc:224] Handling call
0709 13:11:33.552271 (+776433us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:34.041424 29083 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.359s	user 0.002s	sys 0.009s
W20260709 13:11:33.968029 31004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.002s	user 0.006s	sys 0.033s
W20260709 13:11:33.953917 29076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.797s	user 0.002s	sys 0.027s
I20260709 13:11:34.009639 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 585) took 2375 ms. Trace:
W20260709 13:11:34.010378 29125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.248s	user 0.008s	sys 0.002s
W20260709 13:11:33.021948 31025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.927s	user 0.006s	sys 0.027s
I20260709 13:11:33.939544 31579 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.058442 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1587) took 1381 ms. Trace:
I20260709 13:11:34.059326 31579 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.058837 29079 rpcz_store.cc:276] 0709 13:11:32.676634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.676761 (+   127us) service_pool.cc:224] Handling call
0709 13:11:32.800728 (+123967us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:32.800987 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:32.801012 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:32.801452 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:11:34.006981 (+1205529us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:34.007035 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.007126 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.058253 (+ 51127us) raft_consensus.cc:1685] finished
0709 13:11:34.058278 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.058389 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":76638,"threads_started":1,"wal-append.queue_time_us":1086112}
I20260709 13:11:34.000375 31170 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.060784 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.061530 29079 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:34.062269 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1570) took 3500 ms. Trace:
I20260709 13:11:34.062350 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:34.063004 29079 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:33.672260 31609 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.994s	user 0.000s	sys 0.001s
I20260709 13:11:33.965276 31496 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:32.970708 31516 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:34.016013 29085 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.331s	user 0.003s	sys 0.008s
I20260709 13:11:34.030836 31266 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:11:34.003099 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:34.026526 29051 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.68:44167 timed out after 1.693s (SENT) [suppressed 3 similar messages]
W20260709 13:11:34.005442 29717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.000s	sys 0.010s
W20260709 13:11:34.072537 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 46036.5us
W20260709 13:11:34.031679 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 62040.6us
W20260709 13:11:34.031822 28975 connection.cc:256] RPC call timeout handler was delayed by -1.0686343509999006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
W20260709 13:11:34.034900 29150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2027) took 1976 ms (client timeout 2575 ms). Trace:
W20260709 13:11:34.074200 29150 rpcz_store.cc:269] 0709 13:11:32.058962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:33.091983 (+1033021us) service_pool.cc:224] Handling call
0709 13:11:34.034839 (+942856us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:32.842649 31529 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.062192 31170 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:33.983560 31510 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:33.670050 31580 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.041033 29082 raft_consensus.cc:614] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:34.042695 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.042346 29153 rpcz_store.cc:276] 0709 13:11:32.827545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.001008 (+1173463us) service_pool.cc:224] Handling call
0709 13:11:34.002202 (+  1194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:32.969866 31448 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.044593 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.047657 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 328) took 2681 ms. Trace:
I20260709 13:11:34.040212 31431 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:34.006886 31628 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.049193 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.045156 29713 rpcz_store.cc:276] 0709 13:11:31.634770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.635426 (+   656us) service_pool.cc:224] Handling call
0709 13:11:31.668953 (+ 33527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:31.669156 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:32.677792 (+1008636us) log.cc:844] Serialized 14 byte log entry
0709 13:11:34.009298 (+1331506us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:34.009371 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:34.009422 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.009478 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.009584 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":338268,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":117,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":5385,"spinlock_wait_cycles":1720576,"thread_start_us":2276089,"threads_started":2,"wal-append.queue_time_us":1295264}
I20260709 13:11:34.048540 31622 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:11:34.003912 31455 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.053681 31626 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:34.106467 31441 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.110035 31529 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:34.110543 31441 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:34.110409 31626 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.106863 31629 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:11:34.111063 31441 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.037485 31524 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:34.055688 31637 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.056025 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2016) took 3342 ms. Trace:
W20260709 13:11:34.056248 31606 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.796s	user 0.011s	sys 0.019s
I20260709 13:11:34.062795 29084 rpcz_store.cc:276] 0709 13:11:30.561531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.667478 (+105947us) service_pool.cc:224] Handling call
0709 13:11:30.685190 (+ 17712us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:30.685540 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:31.139833 (+454293us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b480015c258. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411a76aaa1 00007f411a761b37 00007f411a7619ca 00007f411a85926f 00007f411a858f37 00007f411a86d55c 00007f411a86d508 00007f411a86d488 00007f411a86d443 00007f411a86c6ec 00007f4119796a96
0709 13:11:31.912524 (+772691us) log.cc:844] Serialized 14 byte log entry
0709 13:11:33.955898 (+2043374us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:33.956502 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:11:33.956660 (+   158us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:33.957193 (+   533us) log.cc:844] Serialized 14 byte log entry
0709 13:11:33.957378 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.957478 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.062086 (+104608us) raft_consensus.cc:1685] finished
0709 13:11:34.062105 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.062219 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193037,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":1867,"raft.queue_time_us":17472,"raft.run_cpu_time_us":2703,"raft.run_wall_time_us":453313,"spinlock_wait_cycles":10629504,"thread_start_us":3172998,"threads_started":2,"wal-append.queue_time_us":2099854}
I20260709 13:11:34.007748 31647 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.020529 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1576) took 3344 ms. Trace:
W20260709 13:11:34.066339 29682 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.473s	user 0.006s	sys 0.015s
I20260709 13:11:34.031118 31340 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:34.120536 29078 rpcz_store.cc:276] 0709 13:11:30.675735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.989215 (+313480us) service_pool.cc:224] Handling call
0709 13:11:31.014578 (+ 25363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:31.025478 (+ 10900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:31.025523 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.026076 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:11:31.968765 (+942689us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:32.014994 (+ 46229us) log.cc:844] Serialized 14 byte log entry
0709 13:11:34.011856 (+1996862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.020240 (+  8384us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.020284 (+    44us) raft_consensus.cc:1685] finished
0709 13:11:34.020298 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.020455 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2858,"prepare.queue_time_us":51764,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":21752,"raft.queue_time_us":136449,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":220,"thread_start_us":2885134,"threads_started":2,"wal-append.queue_time_us":1493030}
I20260709 13:11:33.929596 31642 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.079000 31260 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.080322 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 404) took 2435 ms. Trace:
I20260709 13:11:34.121831 31626 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:34.081722 29157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1251) took 2609 ms (client timeout 1658 ms). Trace:
I20260709 13:11:34.021160 31352 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
W20260709 13:11:34.025645 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80389.6us
I20260709 13:11:34.000550 31526 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.089887 31588 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.125416 31260 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.093083 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42002 (request call id 21) took 1409 ms. Trace:
I20260709 13:11:34.092859 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1269) took 2066 ms. Trace:
I20260709 13:11:34.131167 29082 rpcz_store.cc:276] 0709 13:11:32.682124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:33.159247 (+477123us) service_pool.cc:224] Handling call
0709 13:11:34.091789 (+932542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:34.050051 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.073908 31512 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.099274 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 302) took 2704 ms. Trace:
I20260709 13:11:34.135643 29151 rpcz_store.cc:276] 0709 13:11:31.395274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.413087 (+ 17813us) service_pool.cc:224] Handling call
0709 13:11:31.413507 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:31.413951 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:31.413993 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.414571 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:11:32.680991 (+1266420us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:32.681043 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.681130 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.099093 (+1417963us) raft_consensus.cc:1685] finished
0709 13:11:34.099111 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.099221 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2281,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":1265993,"threads_started":1,"wal-append.queue_time_us":2167190}
I20260709 13:11:34.103469 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:34.103678 29155 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:34.103768 29156 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:34.103694 29079 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:34.137367 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2035) took 1817 ms. Trace:
I20260709 13:11:34.108532 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.104272 31431 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.096944 31579 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.113583 29154 rpcz_store.cc:276] 0709 13:11:30.713609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.729298 (+ 15689us) service_pool.cc:224] Handling call
0709 13:11:30.780812 (+ 51514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:30.781240 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:31.912308 (+1131068us) log.cc:844] Serialized 14 byte log entry
0709 13:11:32.710769 (+798461us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:32.801452 (+ 90683us) log.cc:844] Serialized 26 byte log entry
0709 13:11:32.801599 (+   147us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:32.801629 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.801707 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.055774 (+1254067us) raft_consensus.cc:1685] finished
0709 13:11:34.055796 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.055970 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":121088,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":10227,"raft.queue_time_us":1177,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":148173,"thread_start_us":1764852,"threads_started":2,"wal-append.queue_time_us":2100013}
I20260709 13:11:34.116317 31637 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.168427 31637 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.026633 31566 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:34.093201 31649 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.170562 31649 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:34.172566 31638 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:11:34.174134 31649 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.174437 31441 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:34.120036 29049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.215s	user 0.006s	sys 0.003s
W20260709 13:11:34.175009 29049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.409s	user 0.006s	sys 0.003s
I20260709 13:11:34.175278 31650 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.175513 29715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 581) took 2579 ms. Trace:
I20260709 13:11:34.097563 31435 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:34.121140 29682 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.528s	user 0.006s	sys 0.015s
I20260709 13:11:34.176349 31533 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:34.176793 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 1.58384e+06us
I20260709 13:11:34.176904 31533 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:34.178741 29682 connection.cc:256] RPC call timeout handler was delayed by -1.3332624149999219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20590000000000003s for a response
I20260709 13:11:34.179059 31533 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:34.179387 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:34.179579 31526 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.179813 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:34.124735 29157 rpcz_store.cc:269] 0709 13:11:31.472735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.526441 (+ 53706us) service_pool.cc:224] Handling call
0709 13:11:34.081501 (+2555060us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54001cfcd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:34.081634 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284766976}
I20260709 13:11:34.180596 29079 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:34.180791 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.181377 29157 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:34.182374 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.182446 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:34.183068 29078 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:34.183631 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.193308 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:34.193779 29157 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:34.194639 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.195128 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:34.195406 29157 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:34.097554 29083 rpcz_store.cc:276] 0709 13:11:31.366738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.370125 (+  3387us) service_pool.cc:224] Handling call
0709 13:11:31.370406 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:31.370639 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:31.471087 (+100448us) log.cc:844] Serialized 14 byte log entry
0709 13:11:31.573856 (+102769us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b18002edec0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:11:34.046990 (+2473134us) log.cc:844] Serialized 14 byte log entry
0709 13:11:34.047209 (+   219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:34.047271 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:34.047337 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.047437 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.047587 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1691,"prepare.queue_time_us":1349618,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":10138,"raft.queue_time_us":342452,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":133513,"spinlock_wait_cycles":35038976,"thread_start_us":2467597,"threads_started":2,"wal-append.queue_time_us":2584969}
I20260709 13:11:34.132102 29081 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:34.176920 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 335) took 2265 ms. Trace:
I20260709 13:11:34.196969 29080 rpcz_store.cc:276] 0709 13:11:31.911788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.150724 (+238936us) service_pool.cc:224] Handling call
0709 13:11:32.150964 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:33.925140 (+1774176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:33.925182 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:33.925240 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:33.925283 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.176708 (+251425us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b5400120a54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:34.176748 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.176858 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1557295360}
I20260709 13:11:34.132277 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:34.134371 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.198583 29010 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:34.198855 29007 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:34.130573 31260 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.136891 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.137696 29155 rpcz_store.cc:276] 0709 13:11:32.319925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.048958 (+1729033us) service_pool.cc:224] Handling call
0709 13:11:34.137314 (+ 88356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:34.135352 31400 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:34.138346 31639 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:34.138677 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.165686 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.180004 29082 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:34.131232 29159 rpcz_store.cc:276] 0709 13:11:32.026672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.042257 (+2015585us) service_pool.cc:224] Handling call
0709 13:11:34.092795 (+ 50538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:34.179425 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.086027 31516 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:34.184698 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 81295.3us
W20260709 13:11:34.179073 29682 connection.cc:256] RPC call timeout handler was delayed by -1.2719492339999761s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
I20260709 13:11:34.185343 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1666) took 2448 ms. Trace:
I20260709 13:11:34.185403 31649 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:34.188539 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:34.183068 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1631) took 5796 ms. Trace:
I20260709 13:11:34.268765 29010 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:34.196456 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.271277 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2031) took 2211 ms. Trace:
I20260709 13:11:34.271736 29157 rpcz_store.cc:276] 0709 13:11:32.090880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.196198 (+2105318us) service_pool.cc:224] Handling call
0709 13:11:34.271201 (+ 75003us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:34.200201 29078 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:34.138942 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.175915 29715 rpcz_store.cc:276] 0709 13:11:31.595769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.596750 (+   981us) service_pool.cc:224] Handling call
0709 13:11:31.605124 (+  8374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:31.644287 (+ 39163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:31.644329 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.668469 (+ 24140us) log.cc:844] Serialized 26 byte log entry
0709 13:11:33.975051 (+2306582us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:33.975091 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.975153 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.175300 (+200147us) raft_consensus.cc:1685] finished
0709 13:11:34.175319 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.175458 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41256,"raft.queue_time_us":45860,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":9843,"thread_start_us":2149776,"threads_started":1,"wal-append.queue_time_us":2392771}
I20260709 13:11:34.268780 29084 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:34.138180 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2028) took 2079 ms. Trace:
I20260709 13:11:34.209690 29158 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:34.230752 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.250108 29151 raft_consensus.cc:2410] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:34.124388 31340 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.251123 31579 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.175796 31650 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:34.225908 29680 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:34.249931 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:34.146893 31249 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:34.281981 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:34.138892 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:34.282928 29009 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:11:34.283236 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1267) took 2315 ms. Trace:
I20260709 13:11:34.283631 29151 rpcz_store.cc:276] 0709 13:11:31.967572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.179170 (+2211598us) service_pool.cc:224] Handling call
0709 13:11:34.283178 (+104008us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:34.292173 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 252765us
I20260709 13:11:34.292565 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.293432 29159 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:34.174973 31448 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.258563 29005 rpcz_store.cc:276] 0709 13:11:31.737173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.739379 (+  2206us) service_pool.cc:224] Handling call
0709 13:11:31.739772 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:31.740022 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:31.740176 (+   154us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.838556 (+ 98380us) log.cc:844] Serialized 26 byte log entry
0709 13:11:32.763973 (+925417us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:32.764012 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.764111 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.185128 (+1421017us) raft_consensus.cc:1685] finished
0709 13:11:34.185152 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.185282 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":850323,"threads_started":1,"wal-append.queue_time_us":2152222}
I20260709 13:11:34.275759 29150 rpcz_store.cc:276] 0709 13:11:32.059262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.093416 (+2034154us) service_pool.cc:224] Handling call
0709 13:11:34.138117 (+ 44701us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:34.276719 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.126597 31654 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.268916 29003 rpcz_store.cc:276] 0709 13:11:28.386790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.396955 (+ 10165us) service_pool.cc:224] Handling call
0709 13:11:28.397251 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:28.631592 (+234341us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:28.641785 (+ 10193us) log.cc:844] Serialized 14 byte log entry
0709 13:11:28.678466 (+ 36681us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.012425 (+2333959us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b2c00000f48. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:31.013249 (+   824us) log.cc:844] Serialized 26 byte log entry
0709 13:11:32.071868 (+1058619us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:32.071909 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.071979 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.055691 (+1983712us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:34.182772 (+127081us) raft_consensus.cc:1685] finished
0709 13:11:34.182791 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.182907 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83380,"prepare.queue_time_us":11172,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":60072,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":33079,"spinlock_wait_cycles":169695232,"thread_start_us":1253228,"threads_started":3,"wal-append.queue_time_us":1358544,"wal-append.run_cpu_time_us":2924,"wal-append.run_wall_time_us":1708971}
I20260709 13:11:34.304762 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:34.278595 29155 raft_consensus.cc:2376] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5. Re-sending same reply.
I20260709 13:11:34.279975 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.280258 29081 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:11:34.123943 29708 rpcz_store.cc:276] 0709 13:11:31.644927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.663239 (+ 18312us) service_pool.cc:224] Handling call
0709 13:11:31.856875 (+193636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:31.876892 (+ 20017us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:33.939843 (+2062951us) log.cc:844] Serialized 14 byte log entry
0709 13:11:34.037130 (+ 97287us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:34.040832 (+  3702us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:34.049220 (+  8388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.065500 (+ 16280us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.080238 (+ 14738us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":398,"prepare.queue_time_us":152621,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":11588,"raft.run_wall_time_us":8978,"spinlock_wait_cycles":40683648,"thread_start_us":2067108,"threads_started":2,"wal-append.queue_time_us":106655}
I20260709 13:11:34.296543 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.320590 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.297072 29082 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:34.297395 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:34.298745 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:34.299508 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 73726.6us
W20260709 13:11:34.322283 28976 connection.cc:256] RPC call timeout handler was delayed by -2.9641465339999513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
I20260709 13:11:34.322353 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:34.323604 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.300815 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:34.324498 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:34.319061 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 272238us
I20260709 13:11:34.254217 31431 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.329239 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.330375 29078 raft_consensus.cc:2410] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:11:34.334925 29084 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:34.336546 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:34.337393 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.337421 29714 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:34.338212 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:34.339152 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.340759 29713 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:34.343327 29005 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:34.345719 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.345959 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.348354 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.349352 29159 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:34.275395 31639 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.273212 31400 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.278553 31649 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.294663 31650 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.359985 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.543812 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:34.544523 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 190938us
I20260709 13:11:34.544952 29003 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:34.545181 29001 raft_consensus.cc:2410] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:11:34.545563 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:34.360425 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.458002 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:34.546502 29081 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:34.544059 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1266) took 2591 ms. Trace:
I20260709 13:11:34.505801 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.561523 29151 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:34.561774 29002 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:34.562229 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1249) took 3122 ms. Trace:
I20260709 13:11:34.321075 31249 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:34.562880 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 489105us
I20260709 13:11:34.543834 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.395941 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:34.564109 29008 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:34.550489 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 92607.2us
W20260709 13:11:34.564639 29048 connection.cc:256] RPC call timeout handler was delayed by -1.0230662030000985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
I20260709 13:11:34.562640 29151 rpcz_store.cc:276] 0709 13:11:31.439791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.366289 (+2926498us) service_pool.cc:224] Handling call
0709 13:11:34.561157 (+194868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.561964 (+   807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.562187 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:34.395499 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:34.549088 29159 rpcz_store.cc:276] 0709 13:11:31.953628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.393674 (+2440046us) service_pool.cc:224] Handling call
0709 13:11:34.394058 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.394348 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.394375 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:34.394414 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:34.394443 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.394498 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.543974 (+149476us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":319,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":8661}
I20260709 13:11:34.396318 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:34.556311 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:34.557202 29155 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:11:34.568910 29155 pending_rounds.cc:85] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 4
I20260709 13:11:34.412730 29082 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:11:34.559234 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:34.527644 31658 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.404s	user 0.001s	sys 0.000s
I20260709 13:11:34.557415 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:34.336549 31526 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:34.346597 31533 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:34.574481 29005 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:34.518240 31605 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:11:34.565663 29076 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.007s	sys 0.008s
W20260709 13:11:34.575126 29076 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.007s	sys 0.008s
I20260709 13:11:34.565052 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:34.474220 29714 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:34.299410 31628 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:34.569674 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:34.569263 29155 pending_rounds.cc:107] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:11:34.578866 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.547367 31529 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:34.570607 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:34.570611 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.588310 29081 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:34.588351 29079 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:34.517987 29009 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:34.430851 29157 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:11:34.573436 29125 connection.cc:256] RPC call timeout handler was delayed by -1.1978219409999156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
I20260709 13:11:34.533370 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.579192 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.502779 31622 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.363184 31512 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:34.579514 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.579742 29155 raft_consensus.cc:2889] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:34.597990 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:34.598763 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1263) took 2729 ms. Trace:
I20260709 13:11:34.599265 29122 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:34.599243 29155 rpcz_store.cc:276] 0709 13:11:31.869783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.378683 (+2508900us) service_pool.cc:224] Handling call
0709 13:11:34.556820 (+178137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.598456 (+ 41636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.598690 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:34.574262 28976 connection.cc:256] RPC call timeout handler was delayed by -2.5520525739999584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
I20260709 13:11:34.413851 31139 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:34.569240 29078 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:34.394143 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.584975 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.551736 31659 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:34.589708 29010 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:34.609198 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1568) took 4056 ms. Trace:
I20260709 13:11:34.609513 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:34.610153 28976 connection.cc:256] RPC call timeout handler was delayed by -1.7068090180000581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
I20260709 13:11:34.578622 31464 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:34.609651 29085 rpcz_store.cc:276] 0709 13:11:30.552845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.642166 (+ 89321us) service_pool.cc:224] Handling call
0709 13:11:30.642481 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:30.643520 (+  1039us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:30.741474 (+ 97954us) log.cc:844] Serialized 14 byte log entry
0709 13:11:30.987563 (+246089us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18003862c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:11:34.067018 (+3079455us) log.cc:844] Serialized 14 byte log entry
0709 13:11:34.084621 (+ 17603us) log.cc:844] Serialized 14 byte log entry
0709 13:11:34.319747 (+235126us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:34.347916 (+ 28169us) log.cc:844] Serialized 26 byte log entry
0709 13:11:34.566730 (+218814us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:34.568012 (+  1282us) log.cc:844] Serialized 14 byte log entry
0709 13:11:34.580641 (+ 12629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.580742 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.609010 (+ 28268us) raft_consensus.cc:1685] finished
0709 13:11:34.609028 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.609144 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34140,"prepare.queue_time_us":1388272,"prepare.run_cpu_time_us":149,"prepare.run_wall_time_us":1900,"raft.queue_time_us":551761,"raft.run_cpu_time_us":535,"raft.run_wall_time_us":46631,"spinlock_wait_cycles":21711488,"thread_start_us":3384054,"threads_started":4,"wal-append.queue_time_us":2338273}
I20260709 13:11:34.589764 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1257) took 2988 ms. Trace:
I20260709 13:11:34.594751 29159 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:34.596292 29150 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:34.587347 31627 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:34.587333 31526 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.601850 29084 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:34.377283 31667 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:34.559595 31441 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:34.592607 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 310662us
W20260709 13:11:34.605332 31678 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:11:34.602463 31450 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:34.656090 29008 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:34.336261 31654 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.612028 28975 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:34.611574 31435 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.587563 31533 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.629091 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:34.670735 29714 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:34.620695 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.620709 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.639642 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1270) took 2591 ms. Trace:
I20260709 13:11:34.640229 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.672307 29150 rpcz_store.cc:276] 0709 13:11:32.048545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.564331 (+2515786us) service_pool.cc:224] Handling call
0709 13:11:34.596111 (+ 31780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.639245 (+ 43134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.639563 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:34.672714 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.620777 29157 rpcz_store.cc:276] 0709 13:11:31.601452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.377894 (+2776442us) service_pool.cc:224] Handling call
0709 13:11:34.430554 (+ 52660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:34.589522 (+158968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.589715 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:34.590247 29125 connection.cc:256] RPC call timeout handler was delayed by -1.0531823080000322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:11:34.625226 31668 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:34.674161 31668 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.648917 31667 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.617283 31449 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.606035 31452 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:34.647745 29683 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:34.657307 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.606025 31622 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.680172 29084 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:34.605463 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:34.659687 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 48440.9us
I20260709 13:11:34.659722 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.603289 31170 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 13:11:34.681516 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 76126.2us
I20260709 13:11:34.660630 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.574220 31249 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.681986 31170 consensus_queue.cc:1243] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:34.681645 28975 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:34.682682 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.683406 29078 raft_consensus.cc:2368] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7.
I20260709 13:11:34.683532 31249 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.684182 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.660418 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.624887 31400 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.685114 29078 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:34.674574 31668 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:34.686682 29050 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:34.686897 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.688880 29078 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:34.690191 31516 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.690735 31516 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:34.691918 29078 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:34.694010 31668 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.694918 31510 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.612571 31139 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:34.696125 31668 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:34.696715 31139 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.679616 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:34.703029 29154 raft_consensus.cc:1813] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:34.625332 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.703931 29154 raft_consensus.cc:2410] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:11:34.676393 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.704809 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.705366 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.641465 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2025) took 2583 ms. Trace:
I20260709 13:11:34.649269 29709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 408) took 2483 ms. Trace:
I20260709 13:11:34.681627 29009 raft_consensus.cc:2410] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:11:34.685587 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.656499 31524 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:34.749454 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:34.673771 29125 connection.cc:256] RPC call timeout handler was delayed by -1.0531823080000322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:11:34.676642 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.675974 29081 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
W20260709 13:11:34.751287 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71734.4us
I20260709 13:11:34.708504 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:34.753044 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:34.669763 31350 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:34.754948 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:34.670059 31450 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:34.761921 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 141338us
I20260709 13:11:34.665561 31583 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.685729 31249 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:34.750268 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.749058 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.658267 31441 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.645982 31627 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.675767 31435 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.706157 29159 rpcz_store.cc:276] 0709 13:11:32.058385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.567602 (+2509217us) service_pool.cc:224] Handling call
0709 13:11:34.594345 (+ 26743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.641176 (+ 46831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.641403 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:34.763582 31170 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:35.319056 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1283) took 1342 ms. Trace:
I20260709 13:11:35.321673 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2036) took 2932 ms. Trace:
W20260709 13:11:35.257246 28975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.009s	sys 0.004s
I20260709 13:11:34.875555 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:35.335163 28975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.017s	sys 0.005s
I20260709 13:11:35.330507 29154 rpcz_store.cc:276] 0709 13:11:33.976178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.343825 (+367647us) service_pool.cc:224] Handling call
0709 13:11:35.318369 (+974544us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54003edad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:35.318749 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308848000}
I20260709 13:11:35.280148 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:35.335721 29157 rpcz_store.cc:276] 0709 13:11:32.389498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.673763 (+2284265us) service_pool.cc:224] Handling call
0709 13:11:35.319414 (+645651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.319985 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:35.320017 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:35.320084 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:35.320145 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.320252 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.321231 (+   979us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b54003edad0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:35.321391 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1402305792}
I20260709 13:11:35.338004 29708 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:34.840691 31450 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:35.339274 31441 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:35.339635 29157 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:35.306562 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:35.269649 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 560976us
I20260709 13:11:34.699962 31449 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:34.770797 31461 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:34.762574 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 407) took 2656 ms. Trace:
W20260709 13:11:35.257817 29050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.004s	sys 0.019s
I20260709 13:11:34.703298 31510 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:34.707634 29709 rpcz_store.cc:276] 0709 13:11:32.162502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.182933 (+ 20431us) service_pool.cc:224] Handling call
0709 13:11:32.252589 (+ 69656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.024799 (+1772210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.610542 (+585743us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b54005f3554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:34.645704 (+ 35162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1318345984}
I20260709 13:11:35.342609 31249 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:35.340008 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:35.345232 29085 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
W20260709 13:11:35.264344 29003 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.007s
W20260709 13:11:35.358657 29050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.009s	sys 0.023s
I20260709 13:11:35.357065 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 307) took 1845 ms. Trace:
I20260709 13:11:34.795475 31672 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:35.361711 29157 rpcz_store.cc:276] 0709 13:11:33.504223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.338709 (+1834486us) service_pool.cc:224] Handling call
0709 13:11:35.339247 (+   538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.347833 (+  8586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.349268 (+  1435us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:35.366648 31435 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.749501 31677 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:11:35.367714 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 686067us
I20260709 13:11:35.366283 29711 rpcz_store.cc:276] 0709 13:11:32.106459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.158661 (+ 52202us) service_pool.cc:224] Handling call
0709 13:11:32.290342 (+131681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:33.951405 (+1661063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.762272 (+810867us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b5400502154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:34.762489 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3818452224}
I20260709 13:11:35.368137 31449 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:35.368739 31672 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:35.369472 31672 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:35.370312 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:35.371377 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:35.274801 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:34.840201 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:35.374236 29003 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.001s	sys 0.012s
I20260709 13:11:35.372314 31449 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:11:35.303634 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:35.378984 29709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 371) took 1055 ms. Trace:
I20260709 13:11:34.953073 31400 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:35.383484 29709 rpcz_store.cc:276] 0709 13:11:34.323684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.369561 (+1045877us) service_pool.cc:224] Handling call
0709 13:11:35.378896 (+  9335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:35.385381 31686 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:34.814215 31649 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:35.385973 31686 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.386466 31686 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:35.386693 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:35.387210 31526 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:35.391952 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 374) took 1015 ms. Trace:
I20260709 13:11:35.392518 29708 rpcz_store.cc:276] 0709 13:11:34.376101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.339515 (+963414us) service_pool.cc:224] Handling call
0709 13:11:35.391862 (+ 52347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:35.395557 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 308) took 1836 ms. Trace:
I20260709 13:11:34.758605 31524 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:35.397428 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 710708us
I20260709 13:11:35.395988 29157 rpcz_store.cc:276] 0709 13:11:33.559058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.372418 (+1813360us) service_pool.cc:224] Handling call
0709 13:11:35.394810 (+ 22392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.395149 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:35.395190 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:35.395261 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:35.395302 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.395362 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.395474 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:35.399941 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:35.401311 31461 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:35.401582 31400 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:35.401518 29709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 376) took 1007 ms. Trace:
I20260709 13:11:35.402367 29709 rpcz_store.cc:276] 0709 13:11:34.394251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.399507 (+1005256us) service_pool.cc:224] Handling call
0709 13:11:35.401445 (+  1938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:35.387912 31435 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:35.404435 31435 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:35.405750 31435 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:35.407423 31435 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:34.709123 31647 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:35.388556 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:35.403641 31583 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:35.410102 29717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 368) took 3688 ms. Trace:
I20260709 13:11:35.410908 31583 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:34.820075 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 305) took 1810 ms. Trace:
I20260709 13:11:35.413702 31524 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:35.415343 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:34.707415 31668 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:35.417774 31449 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:35.410859 31249 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.807785 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:34.818884 31670 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:11:34.836179 31533 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:35.403242 31400 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:34.837044 31639 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:35.384222 31441 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:34.836357 31667 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:34.837679 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:35.481139 29076 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.003s	sys 0.022s
I20260709 13:11:35.415948 31461 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:35.478641 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 202220us
I20260709 13:11:35.485523 29150 rpcz_store.cc:276] 0709 13:11:33.009437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.813046 (+1803609us) service_pool.cc:224] Handling call
0709 13:11:34.819499 (+  6453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.819689 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.819717 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:34.819762 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:34.819798 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.819856 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.819968 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:35.486501 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 609102us
I20260709 13:11:35.488382 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 332) took 4064 ms. Trace:
I20260709 13:11:35.491901 31249 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:35.492410 31249 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.490842 29076 rpcz_store.cc:276] 0709 13:11:31.421446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.147259 (+725813us) service_pool.cc:224] Handling call
0709 13:11:32.147480 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:32.147647 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:32.157165 (+  9518us) log.cc:844] Serialized 14 byte log entry
0709 13:11:34.956316 (+2799151us) log.cc:844] Serialized 14 byte log entry
0709 13:11:35.485612 (+529296us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:35.485737 (+   125us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:35.485860 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.485994 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.486190 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3123,"prepare.queue_time_us":24292,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":2859,"raft.queue_time_us":446457,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":257,"thread_start_us":3323909,"threads_started":4,"wal-append.queue_time_us":1908504}
I20260709 13:11:35.492779 31249 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:35.496127 31654 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:35.496307 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 81256.1us
I20260709 13:11:35.496915 31687 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:35.410583 29717 rpcz_store.cc:276] 0709 13:11:31.721554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.726151 (+  4597us) service_pool.cc:224] Handling call
0709 13:11:31.825724 (+ 99573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:32.816801 (+991077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:32.816845 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:32.835034 (+ 18189us) log.cc:844] Serialized 26 byte log entry
0709 13:11:34.072522 (+1237488us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:34.072561 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.409288 (+1336727us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b5400577bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:35.409378 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:35.409476 (+    98us) raft_consensus.cc:1685] finished
0709 13:11:35.409494 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.409919 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278404480,"thread_start_us":956442,"threads_started":1,"wal-append.queue_time_us":961611,"wal-append.run_cpu_time_us":1812,"wal-append.run_wall_time_us":1002597}
I20260709 13:11:35.484376 29709 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:35.499006 31622 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:35.487326 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 618242us
I20260709 13:11:35.497457 31687 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:35.507476 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 669675us
I20260709 13:11:35.504135 31687 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:35.513412 31449 raft_consensus.cc:2764] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:11:35.513665 31348 consensus_queue.cc:1059] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:11:35.516969 31449 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:35.517943 31348 raft_consensus.cc:3055] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:35.505066 31668 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:35.518380 31348 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:35.519955 31348 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:35.489316 31628 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:35.526710 31622 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:35.531884 29709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 377) took 1102 ms. Trace:
I20260709 13:11:35.521878 31686 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:35.536370 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1169) took 1278 ms. Trace:
I20260709 13:11:35.536586 29709 rpcz_store.cc:276] 0709 13:11:34.429871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.403437 (+973566us) service_pool.cc:224] Handling call
0709 13:11:35.403970 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.505663 (+101693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.531524 (+ 25861us) spinlock_profiling.cc:243] Waited 70.6 ms on lock 0x7b54000154d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:35.531780 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162289408}
I20260709 13:11:35.538173 31686 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:35.536769 29080 rpcz_store.cc:276] 0709 13:11:34.257875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.258623 (+   748us) service_pool.cc:224] Handling call
0709 13:11:34.258926 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:34.259233 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.259259 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:34.279849 (+ 20590us) log.cc:844] Serialized 26 byte log entry
0709 13:11:35.313876 (+1034027us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b180032e5a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:11:35.396335 (+ 82459us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:35.396421 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.396566 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:35.535878 (+139312us) raft_consensus.cc:1685] finished
0709 13:11:35.535898 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.536195 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":759,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":12752,"spinlock_wait_cycles":12374272,"thread_start_us":60183,"threads_started":1,"wal-append.queue_time_us":1244287}
I20260709 13:11:35.538378 31455 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:35.538534 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:35.541179 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1171) took 1282 ms. Trace:
I20260709 13:11:35.541697 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:35.543022 29007 raft_consensus.cc:2410] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:35.541563 29083 rpcz_store.cc:276] 0709 13:11:34.258474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.259615 (+  1141us) service_pool.cc:224] Handling call
0709 13:11:34.259795 (+   180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.328610 (+ 68815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.328645 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:34.328691 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:34.328724 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.540735 (+1212011us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b54000c85d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:35.540796 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.540973 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157804288}
I20260709 13:11:35.539844 31628 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:35.545454 29152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1271) took 3488 ms (client timeout 1897 ms). Trace:
W20260709 13:11:35.545904 29152 rpcz_store.cc:269] 0709 13:11:32.057370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.684420 (+627050us) service_pool.cc:224] Handling call
0709 13:11:35.545037 (+2860617us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400055854. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:35.545232 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2929701120}
I20260709 13:11:35.548080 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 310) took 1928 ms. Trace:
I20260709 13:11:35.550858 31628 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:35.548581 29159 rpcz_store.cc:276] 0709 13:11:33.559741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.472634 (+1912893us) service_pool.cc:224] Handling call
0709 13:11:35.485735 (+ 13101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.486116 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:35.486142 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:35.486182 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:35.486212 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.486266 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.488184 (+  1918us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:35.553258 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44264.2us
I20260709 13:11:35.520912 31668 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:35.555903 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:35.557489 31455 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:35.256225 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:35.569109 29159 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:35.569936 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:35.573794 31512 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:35.574467 31512 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:35.549151 29125 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:35.570000 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1278) took 1595 ms. Trace:
I20260709 13:11:35.580870 29159 rpcz_store.cc:276] 0709 13:11:33.974902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.551839 (+1576937us) service_pool.cc:224] Handling call
0709 13:11:35.552651 (+   812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.569629 (+ 16978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.569918 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:35.580552 31455 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:35.588685 31455 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:35.599054 29158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1265) took 3717 ms (client timeout 2209 ms). Trace:
I20260709 13:11:35.605334 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 309) took 2045 ms. Trace:
I20260709 13:11:35.605885 29157 rpcz_store.cc:276] 0709 13:11:33.559436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.398728 (+1839292us) service_pool.cc:224] Handling call
0709 13:11:35.399190 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.520826 (+121636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:35.520858 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:35.520920 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:35.520990 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.605010 (+ 84020us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54004010d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:35.605062 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.605241 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279143680}
I20260709 13:11:35.576545 31677 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:35.846096 31677 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:35.851208 31672 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:35.855749 31672 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:35.864308 31687 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:35.866870 31687 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:35.916585 29085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1181) took 1444 ms (client timeout 1757 ms). Trace:
W20260709 13:11:35.932097 29085 rpcz_store.cc:269] 0709 13:11:34.472387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.620491 (+148104us) service_pool.cc:224] Handling call
0709 13:11:35.916261 (+1295770us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b540034ec54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:35.916442 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1547051264}
W20260709 13:11:35.939966 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 414864us
I20260709 13:11:35.941416 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:35.950424 31435 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:35.950881 31435 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.951247 31435 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:35.961508 31687 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:35.974282 31687 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:35.976198 31705 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:35.977789 31705 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:35.978250 31705 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:35.973456 31435 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:35.979615 31705 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:35.981436 31705 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:35.989393 31435 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:35.989527 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:35.992190 29709 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:35.996562 29711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 379) took 1493 ms (client timeout 1903 ms). Trace:
W20260709 13:11:36.004881 29711 rpcz_store.cc:269] 0709 13:11:34.503471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.369243 (+865772us) service_pool.cc:224] Handling call
0709 13:11:35.996355 (+627112us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54005f3554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:35.996500 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1431927040}
I20260709 13:11:36.002993 29157 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:36.007238 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1282) took 2030 ms. Trace:
I20260709 13:11:36.008128 29157 rpcz_store.cc:276] 0709 13:11:33.975936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.607111 (+1631175us) service_pool.cc:224] Handling call
0709 13:11:35.607577 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.006253 (+398676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.006485 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.013126 29709 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:11:36.026270 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:36.039094 31249 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.002s
I20260709 13:11:36.043195 31249 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.045467 31249 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:36.049294 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1592) took 2020 ms. Trace:
I20260709 13:11:36.060904 29077 rpcz_store.cc:276] 0709 13:11:34.028982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.104578 (+ 75596us) service_pool.cc:224] Handling call
0709 13:11:34.115628 (+ 11050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:34.558384 (+442756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.558413 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:34.558829 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 13:11:34.588841 (+ 30012us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:34.588890 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.048700 (+1459810us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54000c99d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:36.048774 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.048837 (+    63us) raft_consensus.cc:1685] finished
0709 13:11:36.048857 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.049123 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":235165,"raft.queue_time_us":234913,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6500,"spinlock_wait_cycles":489874304,"thread_start_us":29694,"threads_started":1,"wal-append.queue_time_us":189569,"wal-append.run_cpu_time_us":5157,"wal-append.run_wall_time_us":1085876}
W20260709 13:11:36.054066 31481 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.002s	sys 0.029s
W20260709 13:11:36.087625 31481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.002s	sys 0.029s
W20260709 13:11:36.057267 31266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.012s	sys 0.005s
W20260709 13:11:36.088230 31266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.013s	sys 0.006s
I20260709 13:11:36.042294 31512 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.090396 31512 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:36.063079 31693 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.091599 31693 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.093973 31693 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.042498 29005 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:11:36.063687 31625 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 2.347s	user 0.007s	sys 0.000s
I20260709 13:11:36.063895 31686 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:36.096486 31686 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.066883 29009 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:36.098739 31686 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:36.098986 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2028) took 1480 ms. Trace:
I20260709 13:11:36.099550 29009 rpcz_store.cc:276] 0709 13:11:34.618827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.066224 (+1447397us) service_pool.cc:224] Handling call
0709 13:11:36.066586 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.098656 (+ 32070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.098919 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.100786 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2026) took 1482 ms. Trace:
I20260709 13:11:36.101174 29005 rpcz_store.cc:276] 0709 13:11:34.618294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.041490 (+1423196us) service_pool.cc:224] Handling call
0709 13:11:36.042036 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:36.100352 (+ 58316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.100731 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.105756 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:36.067401 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:36.108029 29083 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:36.069247 31400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.007s	sys 0.011s
W20260709 13:11:36.113701 31400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.008s	sys 0.011s
I20260709 13:11:36.073901 31642 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:36.082682 29050 connection.cc:256] RPC call timeout handler was delayed by -1.2487517230000549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:11:36.115000 31682 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.296s	user 0.001s	sys 0.000s
W20260709 13:11:36.055414 29152 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.016s	sys 0.014s
W20260709 13:11:36.115556 29152 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.017s	sys 0.015s
I20260709 13:11:36.117090 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1685) took 1651 ms. Trace:
I20260709 13:11:36.117266 29007 raft_consensus.cc:1217] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-4.3]   Dedup: 2.2->[4.3-4.3]
I20260709 13:11:36.117830 29007 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:11:36.117518 29003 rpcz_store.cc:276] 0709 13:11:34.465625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.558717 (+ 93092us) service_pool.cc:224] Handling call
0709 13:11:34.569785 (+ 11068us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:34.580140 (+ 10355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.580169 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:34.604565 (+ 24396us) log.cc:844] Serialized 52 byte log entry
0709 13:11:34.677051 (+ 72486us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:35.388811 (+711760us) log.cc:844] Serialized 14 byte log entry
0709 13:11:35.411806 (+ 22995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.411922 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.114269 (+702347us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:36.116730 (+  2461us) raft_consensus.cc:1685] finished
0709 13:11:36.116830 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.117013 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45672,"prepare.queue_time_us":676393,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":79,"raft.queue_time_us":33028,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":1880,"spinlock_wait_cycles":41186048,"thread_start_us":703572,"threads_started":2,"wal-append.queue_time_us":213653}
W20260709 13:11:36.114950 29050 connection.cc:256] RPC call timeout handler was delayed by -1.1340129000001298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
I20260709 13:11:36.122331 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:36.128671 29077 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:36.126363 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2030) took 1507 ms. Trace:
I20260709 13:11:36.130125 29009 rpcz_store.cc:276] 0709 13:11:34.619367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.100575 (+1481208us) service_pool.cc:224] Handling call
0709 13:11:36.100913 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.101271 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:36.101572 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 13:11:36.125949 (+ 24377us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:36.126010 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:36.126081 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.126162 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.126297 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6,"prepare.queue_time_us":1254,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":2163,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":23872,"threads_started":1,"wal-append.queue_time_us":12625}
I20260709 13:11:36.125295 29085 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:36.134229 31442 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.127337 29153 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:36.134429 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:36.150099 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 315) took 1968 ms. Trace:
I20260709 13:11:36.150303 29078 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:36.136916 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:36.150444 29153 rpcz_store.cc:276] 0709 13:11:34.181557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.126681 (+1945124us) service_pool.cc:224] Handling call
0709 13:11:36.127024 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.149787 (+ 22763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.150039 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.151340 29076 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:36.137892 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:36.152727 31442 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:36.153172 31442 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:36.153501 31442 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.154091 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1667) took 1859 ms. Trace:
I20260709 13:11:36.154472 29010 rpcz_store.cc:276] 0709 13:11:34.294747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.296132 (+  1385us) service_pool.cc:224] Handling call
0709 13:11:34.589482 (+293350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.609335 (+ 19853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.153690 (+1544355us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400184cd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:36.153900 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581175552}
I20260709 13:11:36.138736 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:36.139969 29077 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:36.146430 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2032) took 1507 ms. Trace:
I20260709 13:11:36.160717 29005 rpcz_store.cc:276] 0709 13:11:34.639096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.101955 (+1462859us) service_pool.cc:224] Handling call
0709 13:11:36.102316 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.102617 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:36.120183 (+ 17566us) log.cc:844] Serialized 14 byte log entry
0709 13:11:36.146074 (+ 25891us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:36.146128 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:36.146169 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.146232 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.146360 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":198,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":111,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":42387,"threads_started":2,"wal-append.queue_time_us":24340}
I20260709 13:11:36.149207 31650 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.176991 31650 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.177733 31650 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:36.214298 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.215507 31650 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.217765 29085 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
W20260709 13:11:36.218561 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 80746.2us
I20260709 13:11:36.219805 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:36.221397 31650 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:36.221905 31650 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:36.222249 31650 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.226651 29050 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:36.227561 31642 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:36.227975 31642 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:36.228353 31642 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.233662 29712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 591) took 2047 ms. Trace:
I20260709 13:11:36.234511 31693 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:36.235201 31693 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:36.248504 29051 leader_election.cc:336] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Timed out: RequestConsensusVote RPC to 127.27.63.68:44167 timed out after 1.688s (SENT)
I20260709 13:11:36.235955 31642 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.238327 31693 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:36.259684 31642 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.230780 31650 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.242357 31442 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.262897 31442 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:36.261251 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.265358 31650 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.265570 29159 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
W20260709 13:11:36.255671 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.266165 31400 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:36.266503 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.266891 31400 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.266180 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1281) took 2290 ms. Trace:
I20260709 13:11:36.268581 29159 rpcz_store.cc:276] 0709 13:11:33.975733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.264859 (+2289126us) service_pool.cc:224] Handling call
0709 13:11:36.265313 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.265960 (+   647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.266132 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.270776 31400 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.280289 31464 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:36.280449 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:36.280920 31464 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.283552 31464 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:36.294042 31657 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 2.204s	user 0.003s	sys 0.000s
W20260709 13:11:35.599671 29158 rpcz_store.cc:269] 0709 13:11:31.882435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.770588 (+888153us) service_pool.cc:224] Handling call
0709 13:11:35.598582 (+2827994us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540013c254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:35.598826 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618411776}
W20260709 13:11:36.307574 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 1.70303e+06us
I20260709 13:11:36.310169 29716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 406) took 4605 ms. Trace:
I20260709 13:11:36.310585 29716 rpcz_store.cc:276] 0709 13:11:31.704196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.704329 (+   133us) service_pool.cc:224] Handling call
0709 13:11:31.742793 (+ 38464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:31.743042 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:31.743079 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:31.819883 (+ 76804us) log.cc:844] Serialized 26 byte log entry
0709 13:11:32.680145 (+860262us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:32.680186 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.680255 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.062909 (+3382654us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:36.309614 (+246705us) raft_consensus.cc:1685] finished
0709 13:11:36.309634 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.309955 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87330048,"thread_start_us":845981,"threads_started":1,"wal-append.queue_time_us":1724154}
I20260709 13:11:36.234061 29712 rpcz_store.cc:276] 0709 13:11:34.186055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.321693 (+135638us) service_pool.cc:224] Handling call
0709 13:11:34.376253 (+ 54560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:34.575425 (+199172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.575470 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:34.575525 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:34.575579 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.233215 (+1657636us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400730354. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:36.233277 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.233490 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457346432}
I20260709 13:11:36.313323 31025 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:36.314043 31025 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:36.321338 28976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.006s	sys 0.008s
W20260709 13:11:36.321730 28976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.007s	sys 0.009s
W20260709 13:11:36.321973 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 808398us
W20260709 13:11:36.325887 29154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.003s	sys 0.011s
W20260709 13:11:36.326210 29154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.003s	sys 0.011s
I20260709 13:11:36.331611 31025 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:36.334724 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:36.335043 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.335139 29082 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:36.335395 29009 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:36.341610 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.341765 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:36.342598 29002 raft_consensus.cc:2410] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:36.323750 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:36.344498 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2025) took 1726 ms. Trace:
I20260709 13:11:36.344995 31025 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:36.344872 29007 rpcz_store.cc:276] 0709 13:11:34.617998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.543772 (+925774us) service_pool.cc:224] Handling call
0709 13:11:36.117020 (+573248us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:36.118339 (+  1319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.344313 (+225974us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b54001d6fd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:36.344442 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302686976}
I20260709 13:11:36.343036 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2047) took 1506 ms. Trace:
I20260709 13:11:36.346060 29002 rpcz_store.cc:276] 0709 13:11:34.836766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.341161 (+1504395us) service_pool.cc:224] Handling call
0709 13:11:36.342985 (+  1824us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.348721 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:36.349534 29008 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:36.349910 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2046) took 1513 ms. Trace:
I20260709 13:11:36.350059 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:36.350255 29008 rpcz_store.cc:276] 0709 13:11:34.836506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.348404 (+1511898us) service_pool.cc:224] Handling call
0709 13:11:36.349861 (+  1457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.350912 29716 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:36.351630 29009 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:36.361060 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.362103 29006 raft_consensus.cc:2410] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:11:36.362571 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2045) took 1526 ms. Trace:
I20260709 13:11:36.363075 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:36.362859 29006 rpcz_store.cc:276] 0709 13:11:34.836246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.515098 (+678852us) service_pool.cc:224] Handling call
0709 13:11:36.362517 (+847419us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.366715 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:36.374306 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:36.375941 31671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.004s	sys 0.006s
W20260709 13:11:36.376379 31671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.973s	user 0.004s	sys 0.006s
I20260709 13:11:36.377409 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.378031 29003 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:36.380375 29082 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:36.387013 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.392223 31442 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:36.394577 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1192) took 1823 ms. Trace:
I20260709 13:11:36.396332 31490 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:36.395033 29081 rpcz_store.cc:276] 0709 13:11:34.571023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.759094 (+188071us) service_pool.cc:224] Handling call
0709 13:11:34.759552 (+   458us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:11:34.772671 (+ 13119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.772721 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:11:34.773637 (+   916us) log.cc:844] Serialized 78 byte log entry
0709 13:11:35.268635 (+494998us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:35.269569 (+   934us) log.cc:844] Serialized 14 byte log entry
0709 13:11:35.353460 (+ 83891us) log.cc:844] Serialized 14 byte log entry
0709 13:11:35.490015 (+136555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.504258 (+ 14243us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.394232 (+889974us) raft_consensus.cc:1685] finished
0709 13:11:36.394250 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.394407 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148807,"prepare.queue_time_us":82350,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":108,"raft.queue_time_us":847344,"raft.run_cpu_time_us":525,"raft.run_wall_time_us":527,"thread_start_us":351125,"threads_started":2,"wal-append.queue_time_us":691031}
I20260709 13:11:36.397431 31490 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:36.397828 31490 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:36.398772 31490 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.401216 31491 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:36.405556 31490 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:36.405761 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.406642 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2024) took 1789 ms. Trace:
I20260709 13:11:36.407007 29001 rpcz_store.cc:276] 0709 13:11:34.617708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.405304 (+1787596us) service_pool.cc:224] Handling call
0709 13:11:36.406583 (+  1279us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:36.408912 29048 connection.cc:256] RPC call timeout handler was delayed by -1.6562437359999649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
I20260709 13:11:36.407733 29683 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:36.413511 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:36.415459 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1274) took 2442 ms. Trace:
I20260709 13:11:36.422098 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:36.422851 29080 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
W20260709 13:11:36.417778 29048 connection.cc:256] RPC call timeout handler was delayed by -1.42421992900006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3668s for a response
W20260709 13:11:36.424281 29048 connection.cc:256] RPC call timeout handler was delayed by -1.1603437359999589s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2723s for a response
I20260709 13:11:36.428536 31491 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:36.429522 31659 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:36.429747 31491 raft_consensus.cc:3055] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:36.430205 31491 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:36.414719 31671 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:11:36.433015 31677 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:36.433790 31671 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:36.434180 31671 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:36.434373 31400 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:36.434808 31400 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:36.435138 31671 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:36.435907 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.415831 29150 rpcz_store.cc:276] 0709 13:11:33.973548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.521275 (+1547727us) service_pool.cc:224] Handling call
0709 13:11:35.533224 (+ 11949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.314581 (+781357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:36.314611 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:36.314657 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:36.314690 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.415253 (+100563us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b5400139f54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:36.415301 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.415409 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":709756,"raft.queue_time_us":730369,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":101489920}
I20260709 13:11:36.435786 31400 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.436443 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:36.437278 29079 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:36.437276 31400 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:36.439185 31728 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:36.440127 31728 consensus_queue.cc:1243] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:11:36.440676 31728 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:36.441009 31491 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.442499 31491 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:36.441015 31728 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:36.449810 31671 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:36.452149 31728 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.453564 31728 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:36.457928 31340 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.476099 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:36.476841 29079 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:36.480204 31490 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:11:36.482789 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 934088us
I20260709 13:11:36.483045 31668 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:36.484889 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:36.480172 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:36.501866 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:36.504149 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:36.506172 29081 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:36.508671 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1285) took 2532 ms. Trace:
I20260709 13:11:36.508616 31579 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.512095 31260 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:36.513423 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1276) took 2539 ms. Trace:
I20260709 13:11:36.509536 31340 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:36.514014 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:36.514708 29079 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:36.523686 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:36.524566 31579 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:36.514969 31529 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.515671 29152 rpcz_store.cc:276] 0709 13:11:33.974261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.546809 (+1572548us) service_pool.cc:224] Handling call
0709 13:11:35.547349 (+   540us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:35.548568 (+  1219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:35.548617 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:35.549325 (+   708us) log.cc:844] Serialized 52 byte log entry
0709 13:11:36.115927 (+566602us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:36.116286 (+   359us) log.cc:844] Serialized 14 byte log entry
0709 13:11:36.513122 (+396836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.513194 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.513237 (+    43us) raft_consensus.cc:1685] finished
0709 13:11:36.513251 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.513372 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":396583,"prepare.queue_time_us":400,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":398474,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":245,"spinlock_wait_cycles":1792,"thread_start_us":566133,"threads_started":1,"wal-append.queue_time_us":354503}
I20260709 13:11:36.522933 31249 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:36.546950 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:36.546624 31529 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:36.553092 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.553335 31452 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:36.554421 31529 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:36.554651 31627 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.555529 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:36.564252 29008 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:11:36.564671 29008 pending_rounds.cc:85] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 6
I20260709 13:11:36.564911 29008 pending_rounds.cc:107] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:11:36.565167 29008 raft_consensus.cc:2889] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:36.565732 29003 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:36.566601 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2033) took 4310 ms. Trace:
I20260709 13:11:36.566951 29155 rpcz_store.cc:276] 0709 13:11:32.256362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.600631 (+2344269us) service_pool.cc:224] Handling call
0709 13:11:34.601014 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:34.611821 (+ 10807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.611849 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:34.614426 (+  2577us) log.cc:844] Serialized 26 byte log entry
0709 13:11:36.413877 (+1799451us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b1800465e20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:11:36.502481 (+ 88604us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:36.502541 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.502659 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.558655 (+ 55996us) raft_consensus.cc:1685] finished
0709 13:11:36.558680 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.566420 (+  7740us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10364,"raft.queue_time_us":10880,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":25139,"spinlock_wait_cycles":7744384,"thread_start_us":56376,"threads_started":1,"wal-append.queue_time_us":1910831}
I20260709 13:11:36.571636 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:36.572284 29714 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:36.584237 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.585112 29004 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:11:36.585521 29004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2049) took 1748 ms (client timeout 2206 ms). Trace:
W20260709 13:11:36.585919 29004 rpcz_store.cc:269] 0709 13:11:34.837415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.583810 (+1746395us) service_pool.cc:224] Handling call
0709 13:11:36.585468 (+  1658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.585877 31340 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:36.586772 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.588701 29008 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:36.584389 31668 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:36.590291 29004 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:11:36.590463 29683 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:36.591853 29002 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:11:36.592397 29002 pending_rounds.cc:85] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:11:36.592643 29002 pending_rounds.cc:107] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:11:36.592927 29002 raft_consensus.cc:2889] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:36.591756 31665 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:36.595461 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 73845.1us
I20260709 13:11:36.599946 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:36.594391 31449 raft_consensus.cc:2764] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:11:36.601707 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:36.600855 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:36.556754 31249 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.609660 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:36.610754 29708 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:36.594842 31665 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:36.612918 31665 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:36.614046 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:36.615517 31674 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:36.616014 31674 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:36.617049 31674 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:36.617836 29156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1261) took 4902 ms (client timeout 1681 ms). Trace:
W20260709 13:11:36.618191 29156 rpcz_store.cc:269] 0709 13:11:31.715534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.883399 (+167865us) service_pool.cc:224] Handling call
0709 13:11:36.617449 (+4734050us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b540013cc54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:36.617656 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4908711680}
I20260709 13:11:36.621512 31665 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.624811 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:36.625998 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:36.626578 29080 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:36.627104 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:36.627806 31639 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:36.630121 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:36.630932 29715 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:36.631744 31671 consensus_queue.cc:1059] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:36.632244 31449 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:36.633294 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:36.634058 31491 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:36.634443 31491 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:36.634801 31491 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.634830 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:36.635679 31490 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:36.510062 29157 rpcz_store.cc:276] 0709 13:11:33.976637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.009009 (+2032372us) service_pool.cc:224] Handling call
0709 13:11:36.009401 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.009711 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:36.010371 (+   660us) log.cc:844] Serialized 14 byte log entry
0709 13:11:36.508292 (+497921us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:36.508345 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:36.508407 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.508475 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.508615 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":473764,"prepare.queue_time_us":186,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":477561,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":21495,"threads_started":1,"wal-append.queue_time_us":17716}
I20260709 13:11:36.641476 31249 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:36.641659 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 375) took 2247 ms. Trace:
I20260709 13:11:36.642292 31627 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:36.642408 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44190.7us
I20260709 13:11:36.643916 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:36.645944 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:36.646394 31348 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.647310 31579 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:36.647418 31340 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:36.648195 31579 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:36.642059 29708 rpcz_store.cc:276] 0709 13:11:34.393945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.393515 (+999570us) service_pool.cc:224] Handling call
0709 13:11:35.394193 (+   678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.611164 (+1216971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.641399 (+ 30235us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54006b53d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:36.641594 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2797200128}
I20260709 13:11:36.649104 31579 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:36.649549 31579 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:36.650408 31579 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.653445 31627 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:36.657088 29080 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:36.653529 29716 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:36.661384 29683 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:36.665069 29051 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:36.663374 31639 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:11:36.668953 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:36.660502 31665 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:36.673785 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.670215 31637 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:36.668283 31584 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:36.671811 31579 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.709935 31584 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:36.711730 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:36.667361 31679 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:11:36.713233 31431 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:36.713634 31431 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:36.713614 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:36.701762 31260 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:36.697959 31637 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:36.718479 31431 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.721364 31260 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:36.724678 31260 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.726584 28976 leader_election.cc:400] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7.
I20260709 13:11:36.726842 28976 leader_election.cc:403] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:36.720486 31637 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:36.727402 28976 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:36.711011 31584 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.727383 31490 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7.
I20260709 13:11:36.722651 31452 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:36.732292 31452 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:36.732649 31452 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.736138 31491 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.738864 31491 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:36.748770 28976 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:36.750186 31490 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:36.750762 31490 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:36.750259 31491 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:36.751204 31490 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:36.751785 31435 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:36.760664 31629 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:36.751085 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:36.764750 31435 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:36.765303 31435 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:36.766691 28906 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 7, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 7 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:36.766590 31435 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.774655 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1273) took 4716 ms. Trace:
W20260709 13:11:36.777951 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 284718us
W20260709 13:11:36.743830 28907 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52676 (request call id 242) took 27116 ms (client timeout 15000 ms). Trace:
W20260709 13:11:36.788997 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.794588 31648 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.795298 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:36.798982 31524 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:36.799489 29157 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:36.799616 31524 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.800779 29156 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:36.801265 31637 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.802313 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.802601 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.803140 29155 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:36.803586 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.803954 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.804379 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:36.804453 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.804603 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:36.805401 29712 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:36.805370 29158 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:36.806994 29150 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:36.804791 29155 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:36.808089 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:36.801862 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.815421 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.801803 31524 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:36.824175 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:36.824904 29004 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:36.825745 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.779747 29151 rpcz_store.cc:276] 0709 13:11:32.058052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.566293 (+2508241us) service_pool.cc:224] Handling call
0709 13:11:34.566620 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.400178 (+833558us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:35.476167 (+ 75989us) log.cc:844] Serialized 14 byte log entry
0709 13:11:35.578292 (+102125us) log.cc:844] Serialized 14 byte log entry
0709 13:11:36.772877 (+1194585us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:36.772926 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:36.772976 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.774256 (+  1280us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54002b5cd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:36.774293 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.774462 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2420527,"prepare.queue_time_us":91821,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":1366254,"raft.run_cpu_time_us":535,"raft.run_wall_time_us":22719,"spinlock_wait_cycles":447941888,"thread_start_us":148339,"threads_started":2,"wal-append.queue_time_us":78161}
I20260709 13:11:36.810218 31637 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:36.833666 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.835115 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:36.836290 29157 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:36.829769 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:36.822510 31699 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.795238 31452 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:36.838702 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43500.6us
I20260709 13:11:36.839682 29155 raft_consensus.cc:2368] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7.
W20260709 13:11:36.839692 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.840519 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:36.841145 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:36.842298 28906 catalog_manager.cc:5697] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:36.838624 31491 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:36.840503 31524 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:36.854305 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.856766 31352 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:36.857445 31352 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.859720 31352 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:36.816910 31648 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:36.860869 28939 debug-util.cc:398] Leaking SignalData structure 0x7b080003a860 after lost signal to thread 27904
I20260709 13:11:36.862231 31400 raft_consensus.cc:3055] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:36.864151 31569 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.865320 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:36.866094 31452 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:36.866884 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71714.2us
I20260709 13:11:36.868525 28906 catalog_manager.cc:5697] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 5, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 5 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.870213 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 306) took 3711 ms. Trace:
I20260709 13:11:36.845023 31490 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.871625 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:36.873715 31733 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.321s	user 0.012s	sys 0.010s
I20260709 13:11:36.874437 29080 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:36.875365 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.855242 31699 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:36.789502 28907 rpcz_store.cc:269] 0709 13:11:09.627416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.631324 (+  3908us) service_pool.cc:224] Handling call
0709 13:11:36.743558 (+27112234us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:30.289602 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:30.298819 (+  9217us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:30.298886 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:11:30.298914 (+    28us) tablet.cc:662] Decoding operations
0709 13:11:30.687308 (+388394us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:30.729606 (+ 42298us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:30.732994 (+  3388us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:11:30.932630 (+199636us) tablet.cc:701] Row locks acquired
0709 13:11:30.932672 (+    42us) write_op.cc:260] PREPARE: finished
0709 13:11:30.932900 (+   228us) write_op.cc:270] Start()
0709 13:11:30.933078 (+   178us) write_op.cc:276] Timestamp: P: 1783602690932815 usec, L: 0
0709 13:11:30.933104 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:11:30.980890 (+ 47786us) log.cc:844] Serialized 4384 byte log entry
0709 13:11:32.665233 (+1684343us) spinlock_profiling.cc:243] Waited 7.95 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:34.141446 (+1476213us) op_driver.cc:464] REPLICATION: finished
0709 13:11:34.315569 (+174123us) write_op.cc:301] APPLY: starting
0709 13:11:34.315670 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:34.315920 (+   250us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:34.315958 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:34.317538 (+  1580us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:34.317829 (+   291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:11:34.317881 (+    52us) write_op.cc:312] APPLY: finished
0709 13:11:34.318295 (+   414us) log.cc:844] Serialized 83 byte log entry
0709 13:11:34.376292 (+ 57997us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:34.417274 (+ 40982us) write_op.cc:454] Released schema lock
0709 13:11:34.428912 (+ 11638us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:34.465401 (+ 36489us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:34.465430 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2054602,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":74261,"apply.run_cpu_time_us":3848,"apply.run_wall_time_us":149806,"num_ops":11,"prepare.queue_time_us":360123,"prepare.run_cpu_time_us":64685,"prepare.run_wall_time_us":1924294,"raft.queue_time_us":153307,"raft.run_cpu_time_us":7824,"raft.run_wall_time_us":206462,"replication_time_us":3187655,"spinlock_wait_cycles":18917888,"thread_start_us":1569487,"threads_started":4,"wal-append.queue_time_us":1328275,"wal-append.run_cpu_time_us":33002,"wal-append.run_wall_time_us":3074329}]]}
I20260709 13:11:36.855074 31266 consensus_queue.cc:1059] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:11:36.850283 31455 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:36.856802 29003 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:36.884232 29712 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:11:36.887776 28939 debug-util.cc:398] Leaking SignalData structure 0x7b080003a580 after lost signal to thread 28942
I20260709 13:11:36.888785 29155 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:36.898599 29157 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:36.888357 31400 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:36.900190 29153 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:36.852298 31491 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:36.900385 31569 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:36.901665 29122 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:36.899341 28906 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 5, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 5 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:36.912884 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:36.909459 28939 debug-util.cc:398] Leaking SignalData structure 0x7b080003aea0 after lost signal to thread 29034
I20260709 13:11:36.913869 29051 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:36.914188 28939 debug-util.cc:398] Leaking SignalData structure 0x7b080003aec0 after lost signal to thread 29035
W20260709 13:11:36.915548 28939 debug-util.cc:398] Leaking SignalData structure 0x7b08003f71c0 after lost signal to thread 29109
W20260709 13:11:36.915907 28939 debug-util.cc:398] Leaking SignalData structure 0x7b08003f71a0 after lost signal to thread 29110
W20260709 13:11:36.917236 28939 debug-util.cc:398] Leaking SignalData structure 0x7b0800362ac0 after lost signal to thread 29183
I20260709 13:11:36.918167 29151 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:36.917589 28939 debug-util.cc:398] Leaking SignalData structure 0x7b0800391140 after lost signal to thread 29184
I20260709 13:11:36.890612 29154 rpcz_store.cc:276] 0709 13:11:33.159138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.336586 (+2177448us) service_pool.cc:224] Handling call
0709 13:11:35.337526 (+   940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.400580 (+ 63054us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:35.483381 (+ 82801us) log.cc:844] Serialized 14 byte log entry
0709 13:11:36.517658 (+1034277us) log.cc:844] Serialized 14 byte log entry
0709 13:11:36.798647 (+280989us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:36.798700 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:36.798747 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.869882 (+ 71135us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b54002b52d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:36.869950 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.870138 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":732843,"prepare.queue_time_us":124707,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":480787,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":2248,"spinlock_wait_cycles":143747840,"thread_start_us":904528,"threads_started":2,"wal-append.queue_time_us":411488}
W20260709 13:11:36.921768 28939 debug-util.cc:398] Leaking SignalData structure 0x7b08003dde00 after lost signal to thread 29741
I20260709 13:11:36.922611 29125 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:36.922976 28939 debug-util.cc:398] Leaking SignalData structure 0x7b08003d6e60 after lost signal to thread 29742
I20260709 13:11:36.928236 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:36.911015 31491 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:36.930872 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:36.897065 31524 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.925222 31455 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:36.935776 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.934078 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.919986 31628 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:36.934930 31352 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:36.941684 31352 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:36.942121 31352 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:36.942905 31668 raft_consensus.cc:3055] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:36.943329 31668 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:36.949025 31400 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:36.926934 31249 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:36.910876 31677 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:11:36.933228 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:36.936508 29154 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:36.960331 28906 catalog_manager.cc:5697] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 6, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 6 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:36.961320 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:36.967171 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:36.962464 28906 catalog_manager.cc:5697] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 3 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:36.968808 31435 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:36.976429 29079 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:11:36.977644 29155 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:36.963472 31266 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:36.963090 31628 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:36.993422 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 97786.1us
I20260709 13:11:36.974639 31435 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:36.978691 31249 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:37.048242 28906 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 5, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 5 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:37.010139 31758 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:37.011232 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.011897 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.014573 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.049711 29155 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:37.023715 29080 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:11:37.025789 29081 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:37.026916 29085 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:37.028136 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:37.053989 29077 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:37.030369 29712 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: [4.5-6.6]
I20260709 13:11:37.032189 29078 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:11:37.032244 31757 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:37.033921 29084 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:37.040077 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:37.043968 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.015756 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.060189 29082 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:11:36.997138 31266 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:36.995780 31629 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:37.067683 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:37.078861 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:37.071583 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.002776 31435 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:37.098111 29050 leader_election.cc:336] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.686s (SENT)
I20260709 13:11:37.104234 29050 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:36.997666 31529 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 13:11:37.100845 28906 catalog_manager.cc:5697] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:37.110023 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:37.065864 31266 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:36.983479 31455 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.077227 31629 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:37.115093 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.117632 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.114557 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.119151 29080 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:37.157794 29077 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:37.158860 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:37.159080 31741 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:11:37.162753 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 76805.8us
I20260709 13:11:36.996219 31668 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:37.165295 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:37.167032 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.116914 31260 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:37.164921 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.120072 31629 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:37.173539 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:37.142439 31490 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:37.180742 31668 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:37.180971 31745 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:11:37.184394 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.155233 31452 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.109117 31431 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:37.194111 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 51245.5us
I20260709 13:11:37.187476 28906 catalog_manager.cc:5697] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 2 to 4, term changed from 1 to 4, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to <none>, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:37.171854 29003 raft_consensus.cc:2393] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:37.194831 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:37.178447 31682 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:11:37.203766 29155 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:37.204689 29080 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:37.207265 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:37.218421 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.201066 31758 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:37.212186 31452 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:37.231349 31747 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:11:37.234738 29157 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:37.235149 29157 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:37.235972 29157 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:37.237047 29157 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:37.237418 31431 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:37.240089 31752 log.cc:927] Time spent T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:11:37.241166 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.244882 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:37.245814 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:37.247431 31449 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:37.247576 31025 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:37.249460 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.250531 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.251569 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.252451 31639 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:37.255086 31452 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:37.256861 31744 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.278s	user 0.003s	sys 0.004s
I20260709 13:11:37.247864 31449 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:37.259174 31449 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:37.259563 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 182431us
I20260709 13:11:37.259878 29006 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:37.260694 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:37.261173 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.261682 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:37.262137 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 155400us
I20260709 13:11:37.262283 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.262724 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.263052 31699 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:37.258459 29159 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:37.263785 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:37.263975 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.202534 31569 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:37.274919 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.283890 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.284729 29003 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:37.285787 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.286698 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.219602 28906 catalog_manager.cc:5697] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 7, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 7 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:37.288200 29004 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:37.289229 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.290098 29004 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:37.293309 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.294492 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:37.296492 29005 raft_consensus.cc:2393] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:37.299667 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:37.300388 29716 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:37.312150 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:37.274277 31524 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:37.324693 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:37.299959 31747 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:11:37.190703 31628 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:37.340569 29003 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:37.337999 28906 catalog_manager.cc:5697] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 6, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 6 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:37.202697 31249 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.344748 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:37.313742 31481 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:37.368645 28906 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:37.365478 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:37.378334 29003 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:37.392443 29085 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:37.402733 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 45012.9us
I20260709 13:11:37.403642 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.419422 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:37.423671 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:37.418143 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.424471 29159 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:37.420590 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:37.427268 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 82589.8us
I20260709 13:11:37.429534 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.433053 28906 catalog_manager.cc:5697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 6, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 6 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:37.436868 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.439036 28906 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:37.345749 31765 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:11:37.445700 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.446641 29152 raft_consensus.cc:2410] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:37.453163 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.389434 31758 consensus_queue.cc:1059] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:11:37.403633 31452 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:11:37.422410 31481 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:37.434397 31348 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:37.468544 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.464954 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.469400 29716 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:37.444900 31675 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:37.427248 31761 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:11:37.479131 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.482465 29156 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:37.495976 31480 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:37.468564 31757 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.508868 29157 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:11:37.509264 29157 pending_rounds.cc:85] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 8
I20260709 13:11:37.509500 29157 pending_rounds.cc:107] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:11:37.509754 29157 raft_consensus.cc:2889] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:37.510257 28906 catalog_manager.cc:5697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:37.507632 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 82470.7us
W20260709 13:11:37.516624 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 78464.6us
I20260709 13:11:37.519193 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.476151 31639 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:37.533486 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.520597 31025 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:37.513543 29708 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:37.539601 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.480113 31490 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 13:11:37.543982 29151 consensus_meta.cc:287] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.000s
I20260709 13:11:37.550326 28906 catalog_manager.cc:5697] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 3, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:37.551828 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:37.558033 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.561003 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.554899 29151 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:37.537050 31025 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:37.569005 28975 leader_election.cc:336] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.589s (SENT)
I20260709 13:11:37.569551 28975 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:37.565529 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:37.566102 28906 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:37.575580 29078 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:37.557070 31490 consensus_queue.cc:1243] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:11:37.517271 31675 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:37.590193 29048 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:37.583801 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 49026.3us
I20260709 13:11:37.583662 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.550633 31765 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:37.596017 29716 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:37.598984 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.562686 31763 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:37.594121 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:37.603309 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:37.589947 28906 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 5, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 5 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:37.605409 29159 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.546867 31757 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:37.609189 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:37.619669 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 50705.8us
W20260709 13:11:37.620910 28975 leader_election.cc:336] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.635s (SENT)
I20260709 13:11:37.553953 31639 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:37.595707 31480 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:37.596850 31765 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:37.622784 29156 raft_consensus.cc:2410] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:37.588536 31773 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:37.622156 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.630609 31674 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:37.660768 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:37.580861 31481 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:37.701524 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 94196.9us
I20260709 13:11:37.636991 31480 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:37.690050 28906 catalog_manager.cc:5697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:37.645987 31772 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:37.732496 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 140417us
I20260709 13:11:37.699784 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:37.697614 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 76752.9us
W20260709 13:11:37.736732 28975 leader_election.cc:336] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.646s (SENT)
I20260709 13:11:37.719419 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.721712 29159 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:37.660665 31348 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:37.718797 31784 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:11:37.731633 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 70922.6us
I20260709 13:11:37.719296 31648 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:37.739467 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:37.742688 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:37.743069 31769 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:11:37.745147 31400 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:37.746220 31761 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:37.751004 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.751461 29082 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:37.751869 29157 raft_consensus.cc:2410] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:37.753364 28906 catalog_manager.cc:5697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 4, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 4 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:37.739244 31668 consensus_meta.cc:287] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.002s
I20260709 13:11:37.706720 31763 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:37.757757 31435 consensus_meta.cc:287] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.002s
I20260709 13:11:37.766731 29155 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:11:37.767087 29155 pending_rounds.cc:85] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 5
I20260709 13:11:37.767292 29155 pending_rounds.cc:107] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:11:37.767493 29155 raft_consensus.cc:2889] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:37.759117 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:37.765383 28906 catalog_manager.cc:5697] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 6, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 6 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:37.758769 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:37.688923 31025 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:37.764568 31491 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.825s	user 0.010s	sys 0.001s
W20260709 13:11:37.756860 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 143312us
I20260709 13:11:37.783594 31763 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.784650 31786 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.786535 31795 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.725807 31765 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.787109 31787 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:37.787369 29002 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:11:37.787582 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 50889.7us
I20260709 13:11:37.788370 31348 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.794116 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:37.808113 28975 leader_election.cc:336] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.649s (SENT)
I20260709 13:11:37.794181 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:37.795179 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:37.808718 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 69309.9us
I20260709 13:11:37.809034 29153 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:37.799144 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:37.804538 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.794364 28906 catalog_manager.cc:5697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 5, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 5 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:37.810181 29713 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 4: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:11:37.787392 31352 consensus_meta.cc:287] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.827s	user 0.008s	sys 0.000s
I20260709 13:11:37.807363 29002 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:37.809706 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:37.848694 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.848852 28906 catalog_manager.cc:5697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 4 to 5, term changed from 2 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66), VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:37.847289 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.812292 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:37.852442 29077 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:37.856410 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 48323.2us
W20260709 13:11:37.857034 28975 leader_election.cc:336] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.658s (SENT)
W20260709 13:11:37.857862 28975 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.681s (SENT)
I20260709 13:11:37.857971 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:37.858033 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:37.858635 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.859298 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:37.860343 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:37.860865 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.795573 31772 raft_consensus.cc:2764] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:11:37.863775 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 129354us
I20260709 13:11:37.865181 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:37.787523 31452 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:37.774580 31435 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.871026 28906 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 3, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:37.876262 29081 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:37.806895 31786 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:37.885403 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.812071 31025 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:37.893769 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:37.894357 29154 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:37.894965 29003 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:37.870317 31628 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:37.867173 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:37.901454 29085 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:37.870456 31796 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:37.904394 29007 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:37.905372 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:37.900144 29077 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:37.797248 31491 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.905965 29081 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:37.863327 31352 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.920751 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:37.922493 29076 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:11:37.904546 31796 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:37.942237 31800 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:11:37.942399 31435 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:37.937819 29010 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:37.893407 31791 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:11:37.956841 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:37.939613 31449 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.680s	user 0.007s	sys 0.003s
I20260709 13:11:37.948333 29077 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:37.935526 31757 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:37.900368 31783 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.165s
W20260709 13:11:38.000967 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 143107us
I20260709 13:11:38.014324 29077 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:37.984649 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:37.959985 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 85112.1us
I20260709 13:11:37.956777 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:37.965909 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.045081 29082 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:37.813513 31795 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.977963 31804 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:37.891362 31480 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:37.964300 31677 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:38.035065 31770 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.484s	user 0.002s	sys 0.000s
I20260709 13:11:37.947602 31796 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:38.053700 28975 leader_election.cc:336] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.897s (SENT)
W20260709 13:11:37.981814 31777 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
W20260709 13:11:37.982690 31266 consensus_meta.cc:287] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.839s	user 0.008s	sys 0.000s
W20260709 13:11:37.968713 31769 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.220s	user 0.001s	sys 0.004s
W20260709 13:11:38.057686 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 133794us
W20260709 13:11:38.065440 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 155639us
I20260709 13:11:37.978626 31491 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.066305 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:37.983757 31352 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:38.002564 31786 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:38.136119 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:38.147017 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 93337.7us
W20260709 13:11:38.148885 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.983939 31675 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:37.984393 31449 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:38.241164 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.299959 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:38.306795 31802 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.263s	user 0.000s	sys 0.000s
W20260709 13:11:38.307950 31788 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 13:11:38.308212 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.309508 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.310418 29076 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:38.310563 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 367) took 1035 ms. Trace:
I20260709 13:11:38.312824 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 368) took 1037 ms. Trace:
I20260709 13:11:38.315687 29080 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:11:38.312841 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:38.327369 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 186884us
I20260709 13:11:38.346123 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.329677 31786 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:38.303301 31025 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:38.353029 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:38.354416 28975 leader_election.cc:336] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 2.209s (SENT)
I20260709 13:11:38.313222 29079 rpcz_store.cc:276] 0709 13:11:37.275454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.275840 (+   386us) service_pool.cc:224] Handling call
0709 13:11:37.291289 (+ 15449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:37.319261 (+ 27972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:37.319290 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:37.343771 (+ 24481us) log.cc:844] Serialized 26 byte log entry
0709 13:11:37.478477 (+134706us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:37.519961 (+ 41484us) log.cc:844] Serialized 14 byte log entry
0709 13:11:37.520448 (+   487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.523067 (+  2619us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:38.307919 (+784852us) raft_consensus.cc:1685] finished
0709 13:11:38.307948 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:38.312767 (+  4819us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39242,"prepare.queue_time_us":54641,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":28905,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":26490,"thread_start_us":73435,"threads_started":1,"wal-append.queue_time_us":221228}
I20260709 13:11:38.350041 31491 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:38.361267 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:38.361388 31803 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.305s	user 0.000s	sys 0.000s
I20260709 13:11:38.361969 29082 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:38.361541 31491 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:38.362762 31805 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 13:11:38.364257 31491 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:38.367766 31797 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:38.368508 31797 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.371106 31797 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.375413 31449 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:38.137365 31677 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.310935 29084 rpcz_store.cc:276] 0709 13:11:37.275170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.275287 (+   117us) service_pool.cc:224] Handling call
0709 13:11:37.275546 (+   259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:37.318903 (+ 43357us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:37.468813 (+149910us) log.cc:844] Serialized 14 byte log entry
0709 13:11:37.609742 (+140929us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:37.629444 (+ 19702us) log.cc:844] Serialized 26 byte log entry
0709 13:11:37.629566 (+   122us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:37.629596 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.629673 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:38.310356 (+680683us) raft_consensus.cc:1685] finished
0709 13:11:38.310379 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:38.310509 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25925,"prepare.queue_time_us":145526,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":20603,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":37094,"thread_start_us":177295,"threads_started":2,"wal-append.queue_time_us":379455}
I20260709 13:11:38.378469 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:38.138616 31266 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.388955 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.390825 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.392861 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:38.393770 29079 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:38.396791 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.387754 29154 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
W20260709 13:11:38.400573 31629 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.191s	user 0.009s	sys 0.003s
I20260709 13:11:38.401110 31629 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.403620 31629 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.404486 29008 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:38.406755 31266 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:38.407437 29158 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:38.353430 31025 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:38.409011 31025 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.410310 31025 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:38.412222 29153 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.004s
I20260709 13:11:38.412665 29153 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:38.414244 29081 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:38.418081 29076 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:38.423843 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.424685 29079 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:38.427385 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.429219 29081 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:38.376626 31786 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.442675 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:38.456310 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 101894us
W20260709 13:11:38.458889 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:38.459649 29125 connection.cc:256] RPC call timeout handler was delayed by -1.8043045889999121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:11:38.459949 29125 connection.cc:256] RPC call timeout handler was delayed by -1.5424045889999434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 13:11:38.460244 29125 connection.cc:256] RPC call timeout handler was delayed by -1.5361045889999332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 13:11:38.460500 29125 connection.cc:256] RPC call timeout handler was delayed by -1.26520458899995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 13:11:38.460752 29125 connection.cc:256] RPC call timeout handler was delayed by -1.1860045889999355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 13:11:38.443338 31629 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:38.440335 31773 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.462754 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.466156 29153 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:38.474015 29004 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:38.510654 29050 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.619s (SENT)
I20260709 13:11:38.511299 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:38.511729 29082 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:38.463938 31266 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.463886 31795 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.546104 31807 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:38.551570 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.552803 29153 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:38.552641 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 87681.8us
I20260709 13:11:38.474112 31529 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.562743 31400 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.563604 31400 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.520083 31675 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:38.553447 29050 leader_election.cc:336] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.830s (SENT)
I20260709 13:11:38.571460 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:38.513146 31773 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.584511 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.585449 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.572791 31435 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.590088 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:38.576112 31584 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.594996 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.595959 29153 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:38.583743 31400 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:38.574723 31529 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.574237 31266 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.582998 31675 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.603811 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:38.604565 31773 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.616307 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.618305 29153 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:38.619750 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:38.620934 31786 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:38.621004 29125 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:38.622828 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:38.623695 31814 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:38.624244 31699 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:38.624472 31814 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:38.624709 31699 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:38.625200 31699 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:38.625732 31677 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.627548 31266 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:38.594790 31435 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.629774 31435 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.631023 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 327) took 1412 ms. Trace:
I20260709 13:11:38.597656 31480 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:38.598029 31584 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.634517 31584 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.639012 31821 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.639544 31773 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:38.625154 31813 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:38.631436 29158 rpcz_store.cc:276] 0709 13:11:37.226074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.259398 (+ 33324us) service_pool.cc:224] Handling call
0709 13:11:37.259685 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:38.407820 (+1148135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:38.630769 (+222949us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400419bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:38.630959 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2639036800}
W20260709 13:11:38.643240 31628 consensus_meta.cc:287] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.733s	user 0.005s	sys 0.005s
I20260709 13:11:38.637516 31807 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:38.685385 31628 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:38.690070 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.690586 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:38.691243 29006 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:38.691586 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:38.692250 29009 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:38.690621 29003 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:38.693991 31675 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.694170 29003 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:38.690747 29082 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:38.697367 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.691267 29007 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:38.700204 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.701390 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:38.706321 29009 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:38.639539 31821 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.737567 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:38.738718 29006 raft_consensus.cc:2410] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:11:38.740875 29158 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:38.749678 31629 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:38.656028 31529 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:38.754645 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.754913 31659 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:38.755527 31659 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.757808 31659 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:38.760911 31796 consensus_meta.cc:287] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.000s
I20260709 13:11:38.761761 31699 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.765406 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:38.766319 29009 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:38.766618 29010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2052) took 1754 ms (client timeout 1780 ms). Trace:
W20260709 13:11:38.767016 29010 rpcz_store.cc:269] 0709 13:11:37.012165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.014232 (+  2067us) service_pool.cc:224] Handling call
0709 13:11:38.766408 (+1752176us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54000af0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:38.766562 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2034171776}
I20260709 13:11:38.761338 31796 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.836690 29001 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
W20260709 13:11:38.836977 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 214194us
I20260709 13:11:38.836220 31629 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.845633 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:38.837455 31813 consensus_queue.cc:1059] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:11:38.873384 31629 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.825492 31481 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:11:38.888515 31671 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.889199 31671 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:38.889741 31796 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:38.891105 31758 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:38.891595 31758 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:38.891824 31671 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:38.892647 31758 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:38.874531 31529 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:11:38.923157 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:39.395341 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 841803us
W20260709 13:11:39.221810 29151 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1307) took 2394 ms (client timeout 1703 ms). Trace:
W20260709 13:11:39.420455 29151 rpcz_store.cc:269] 0709 13:11:36.827416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.830523 (+  3107us) service_pool.cc:224] Handling call
0709 13:11:39.221534 (+2391011us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x7b54003f6bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:39.221716 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169823104}
I20260709 13:11:39.224462 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1712) took 1177 ms. Trace:
W20260709 13:11:39.214552 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.421769 29001 rpcz_store.cc:276] 0709 13:11:38.047163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.047274 (+   111us) service_pool.cc:224] Handling call
0709 13:11:38.058059 (+ 10785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:38.844391 (+786332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.224147 (+379756us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b5400077e54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:39.224387 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790046080}
I20260709 13:11:39.264653 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1608) took 1456 ms. Trace:
W20260709 13:11:39.423144 29050 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.918s (SENT) [suppressed 62 similar messages]
I20260709 13:11:39.422827 29080 rpcz_store.cc:276] 0709 13:11:37.808361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.856384 (+ 48023us) service_pool.cc:224] Handling call
0709 13:11:38.315334 (+458950us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:38.320098 (+  4764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.264412 (+944314us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b5400148a50. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:39.264584 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1054601088}
I20260709 13:11:39.402359 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:39.425032 29076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.008s	sys 0.012s
I20260709 13:11:38.940649 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:39.260612 29008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2053) took 2248 ms (client timeout 1918 ms). Trace:
I20260709 13:11:39.051226 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 324) took 1902 ms. Trace:
I20260709 13:11:39.401343 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:39.104445 31758 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:39.261439 29003 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:39.157411 28975 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.536s (SENT)
I20260709 13:11:39.010541 31452 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:39.401752 29005 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:39.396762 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:39.327206 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:39.316612 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:39.217751 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.266898 29156 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:11:39.215612 29002 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.589836 29077 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:39.549204 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1710) took 1795 ms. Trace:
I20260709 13:11:39.569283 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:39.648983 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.649222 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.425515 29159 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:39.650090 29002 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:39.639695 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1614) took 1752 ms. Trace:
W20260709 13:11:38.938748 31776 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.342s	user 0.008s	sys 0.001s
I20260709 13:11:39.564482 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:39.537989 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.569406 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.553042 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.664718 29157 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:39.567888 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:39.660035 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:39.425472 29076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.008s	sys 0.012s
I20260709 13:11:39.660360 29155 rpcz_store.cc:276] 0709 13:11:37.149159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.242153 (+ 92994us) service_pool.cc:224] Handling call
0709 13:11:37.242444 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:37.767838 (+525394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.050926 (+1283088us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54003ecbd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:39.051150 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204952960}
I20260709 13:11:39.660006 29077 rpcz_store.cc:276] 0709 13:11:37.887689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.566647 (+1678958us) service_pool.cc:224] Handling call
0709 13:11:39.571440 (+  4793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:39.639443 (+ 68003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.639641 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:39.489423 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.663019 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:39.648983 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.663242 29081 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:39.585819 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:39.488644 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.473141 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.639719 29004 rpcz_store.cc:276] 0709 13:11:37.754256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.754388 (+   132us) service_pool.cc:224] Handling call
0709 13:11:37.770911 (+ 16523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:38.474492 (+703581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.548972 (+1074480us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54001851d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:39.549135 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616398080}
W20260709 13:11:39.672749 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.667222 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.684294 29082 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:39.516574 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1612) took 1644 ms. Trace:
I20260709 13:11:39.475437 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:39.666815 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:39.695310 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 219951us
I20260709 13:11:39.684774 29001 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:39.665508 29716 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:39.695938 29010 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:39.665637 29153 raft_consensus.cc:2410] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:39.665479 29150 raft_consensus.cc:2410] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:39.697764 29154 raft_consensus.cc:2410] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:11:39.676616 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 519195us
W20260709 13:11:39.655987 29008 rpcz_store.cc:269] 0709 13:11:37.012575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.213937 (+201362us) service_pool.cc:224] Handling call
0709 13:11:39.260287 (+2046350us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003aaf54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:39.260516 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2673119616}
I20260709 13:11:39.666683 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.698959 29006 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:11:39.610307 29682 leader_election.cc:336] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.550s (SENT)
I20260709 13:11:39.701025 29002 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:11:39.701468 29002 pending_rounds.cc:85] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:11:39.701504 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2088) took 1001 ms. Trace:
I20260709 13:11:39.701893 29002 pending_rounds.cc:107] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:11:39.702263 29002 raft_consensus.cc:2889] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:39.697230 29007 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:39.699394 29078 rpcz_store.cc:276] 0709 13:11:37.874611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.911218 (+ 36607us) service_pool.cc:224] Handling call
0709 13:11:38.940435 (+1029217us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:11:39.516501 (+576066us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2686336}
I20260709 13:11:39.702589 29151 raft_consensus.cc:2376] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6. Re-sending same reply.
I20260709 13:11:39.705466 31025 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.706010 31025 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:39.707487 31826 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:11:38.930467 31529 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:38.930649 31699 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:39.700183 29682 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:39.666522 31828 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:11:39.715212 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2080) took 2360 ms. Trace:
I20260709 13:11:39.512804 31348 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.713037 29716 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:39.738507 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.739069 29159 raft_consensus.cc:2376] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6. Re-sending same reply.
I20260709 13:11:39.588622 31533 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:39.534827 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:39.751070 31025 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:39.751526 29709 raft_consensus.cc:2410] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:11:39.533459 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:39.702066 29154 rpcz_store.cc:276] 0709 13:11:38.721472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.721700 (+   228us) service_pool.cc:224] Handling call
0709 13:11:39.701446 (+979746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:39.213182 31665 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.733701 29002 rpcz_store.cc:276] 0709 13:11:37.354912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.379516 (+ 24604us) service_pool.cc:224] Handling call
0709 13:11:37.422660 (+ 43144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:37.787274 (+364614us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b54003d61a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157b8dc0 00007f41158431c0 00007f4115844231 00007f4115840d8f 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c
0709 13:11:39.215126 (+1427852us) spinlock_profiling.cc:243] Waited 8.17 ms on lock 0x7b4c004368c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f4113b0da20 00007f4113b1ea29 00007f4113ae4644 00007f4113b0bf2d 00007f4113b0bb93 00007f411583c18b 00007f411583ccb5 00007f4115837f30 00007f41158397de 00007f4115843b49 00007f4115844416
0709 13:11:39.714700 (+499574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.715142 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99252224}
I20260709 13:11:39.715471 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:39.755409 29084 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:39.735100 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:39.736326 29081 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:39.756603 29077 raft_consensus.cc:2410] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:11:39.710944 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:39.708509 31648 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:39.708201 31435 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.400756 31629 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:39.571419 31352 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:39.744346 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.744473 29157 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
W20260709 13:11:39.758044 29157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2067) took 1910 ms (client timeout 1754 ms). Trace:
W20260709 13:11:39.745217 31767 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:11:39.599263 31823 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:39.513123 31431 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.497385 31778 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:39.547369 31821 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:39.755728 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.776893 29151 raft_consensus.cc:2376] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4. Re-sending same reply.
I20260709 13:11:39.571846 31812 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:39.758458 29157 rpcz_store.cc:269] 0709 13:11:37.847974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.315439 (+1467465us) service_pool.cc:224] Handling call
0709 13:11:39.757977 (+442538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:39.779942 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.803359 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.804253 29159 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:11:39.806416 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 409682us
I20260709 13:11:39.808769 31843 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:39.809279 31843 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:39.809088 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 198788us
I20260709 13:11:39.809906 31843 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:39.810981 31839 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:39.811697 31839 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.569871 31824 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.206s
I20260709 13:11:39.764917 31481 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:11:39.766607 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 180870us
I20260709 13:11:39.775717 29082 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:39.817529 29125 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:39.817957 29082 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:11:39.709414 31452 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.761310 31665 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.823035 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:39.593773 31639 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:39.767079 31435 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.815279 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.758564 29708 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:39.818589 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.802685 29150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:39.813762 31821 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:39.714803 31455 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:11:39.804713 31431 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.760511 31249 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:11:39.827625 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.758227 31533 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:39.850612 29002 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:39.772256 31629 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:39.817727 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.829649 29151 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:39.830358 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:39.753304 31348 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:39.831228 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:39.815177 31789 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 2.099s	user 0.008s	sys 0.010s
I20260709 13:11:39.815913 31529 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:39.832569 31679 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:39.884707 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.834542 29157 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
W20260709 13:11:39.812669 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 389499us
I20260709 13:11:39.814255 31765 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:39.758343 29079 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:39.885344 29010 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:39.799954 31823 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.814378 31352 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:39.854786 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2069) took 1946 ms. Trace:
W20260709 13:11:39.866894 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.868495 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1722) took 1179 ms. Trace:
I20260709 13:11:39.896835 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.895000 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.898363 29159 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:39.894896 29156 rpcz_store.cc:276] 0709 13:11:37.908391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.908529 (+   138us) service_pool.cc:224] Handling call
0709 13:11:39.225004 (+1316475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:39.656611 (+431607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.854592 (+197981us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003f6bd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:39.854729 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3026654592}
I20260709 13:11:39.900880 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.901463 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.871325 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1720) took 1182 ms. Trace:
W20260709 13:11:39.872408 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.872748 31834 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:39.874691 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:39.880995 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 175927us
W20260709 13:11:39.907819 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.907892 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.902163 29158 raft_consensus.cc:2376] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6. Re-sending same reply.
I20260709 13:11:39.882634 29709 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:39.883255 31845 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:39.828701 31025 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:39.829646 31699 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:39.833873 29150 raft_consensus.cc:606] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:39.760324 31648 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.849835 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:39.903908 29005 rpcz_store.cc:276] 0709 13:11:38.688869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.689128 (+   259us) service_pool.cc:224] Handling call
0709 13:11:39.395018 (+705890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:39.661912 (+266894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.871077 (+209165us) spinlock_profiling.cc:243] Waited 78.6 ms on lock 0x7b54000af0d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:39.871255 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180726656}
I20260709 13:11:39.895754 29003 rpcz_store.cc:276] 0709 13:11:38.689422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.694887 (+  5465us) service_pool.cc:224] Handling call
0709 13:11:39.261247 (+566360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:39.663967 (+402720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.868215 (+204248us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b54003aaf50. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:39.868418 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1301734272}
I20260709 13:11:39.906343 29152 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:39.813596 31778 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.894845 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.908560 29010 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
W20260709 13:11:39.908463 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.917013 29151 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:39.916805 29150 raft_consensus.cc:614] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:39.917886 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.918045 29001 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:39.911520 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.910645 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.812228 31839 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:39.911069 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 80783us
I20260709 13:11:39.813037 31812 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.910192 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1233) took 1509 ms. Trace:
I20260709 13:11:39.919661 29159 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 5: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:39.909155 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:39.920585 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 184949us
I20260709 13:11:39.924865 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:39.919639 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.930393 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:39.839967 31821 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.931810 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.933027 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:39.929895 29152 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:39.907917 31529 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:39.950553 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.942327 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:39.951066 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 143143us
I20260709 13:11:39.942728 29716 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:39.948758 29003 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:39.952296 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.894400 31629 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:39.922737 31648 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:39.956092 31699 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:39.957197 31629 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:39.957854 31839 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:39.958242 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.959335 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.960273 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.961201 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.922086 31480 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:39.926733 29076 rpcz_store.cc:276] 0709 13:11:38.400927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.463960 (+ 63033us) service_pool.cc:224] Handling call
0709 13:11:38.520848 (+ 56888us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:38.830388 (+309540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:38.830438 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:38.831151 (+   713us) log.cc:844] Serialized 52 byte log entry
0709 13:11:39.699200 (+868049us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:39.764238 (+ 65038us) log.cc:844] Serialized 14 byte log entry
0709 13:11:39.779008 (+ 14770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.901982 (+122974us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b5400404a54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:39.902049 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:39.902110 (+    61us) raft_consensus.cc:1685] finished
0709 13:11:39.910007 (+  7897us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:39.910143 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10269,"prepare.queue_time_us":120468,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":54437,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":12679,"spinlock_wait_cycles":173424896,"thread_start_us":1027757,"threads_started":3,"wal-append.queue_time_us":671710}
I20260709 13:11:39.948990 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.931000 29077 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:39.921674 31455 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:39.948908 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:39.965517 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.965725 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.962707 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.961705 29156 raft_consensus.cc:2368] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7.
I20260709 13:11:39.961346 31821 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:39.981169 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.981871 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:39.941545 31249 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:39.984395 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:39.987046 29050 connection.cc:256] RPC call timeout handler was delayed by -1.1715857120000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
W20260709 13:11:39.990018 29050 leader_election.cc:336] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 2.158s (SENT)
I20260709 13:11:39.990545 29050 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:39.990952 29152 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:39.987661 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:39.991315 29152 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:39.991700 29159 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:39.941675 31825 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:39.988334 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 70935.9us
I20260709 13:11:39.988703 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 366) took 2714 ms. Trace:
I20260709 13:11:39.996289 29154 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:39.954661 31824 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:39.926335 31025 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:39.971479 31455 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:40.000260 29716 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: [4.5-6.6]
I20260709 13:11:40.000344 29005 raft_consensus.cc:3055] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:40.005322 29001 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:40.005635 29005 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:39.995833 29083 rpcz_store.cc:276] 0709 13:11:37.274122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.274278 (+   156us) service_pool.cc:224] Handling call
0709 13:11:37.274643 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:37.467611 (+192968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:37.467762 (+   151us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:37.503157 (+ 35395us) log.cc:844] Serialized 26 byte log entry
0709 13:11:37.609412 (+106255us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:37.609446 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.609506 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:39.814908 (+2205402us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b24000fea38. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f41125da9c4 00007f41125d8acb 00007f4112793dc9 00007f4112793a55 00007f411279309d 00007f41153c510e 00007f41153dc88c 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221
0709 13:11:39.983563 (+168655us) raft_consensus.cc:1685] finished
0709 13:11:39.983592 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:39.988646 (+  5054us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30503,"raft.queue_time_us":46510,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":9122,"spinlock_wait_cycles":21063040,"thread_start_us":206717,"threads_started":2,"wal-append.queue_time_us":158682}
I20260709 13:11:39.997712 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:39.978543 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:40.012956 31823 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:39.989121 31699 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:40.003957 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.007699 29154 raft_consensus.cc:2376] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6. Re-sending same reply.
I20260709 13:11:40.008322 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:40.009295 29010 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:40.002765 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:40.016538 29050 connection.cc:256] RPC call timeout handler was delayed by -1.0507516099999066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:11:40.018551 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 168074us
I20260709 13:11:40.024176 29155 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:11:40.024343 29050 connection.cc:256] RPC call timeout handler was delayed by -1.0183516099999679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 13:11:39.995286 31480 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:40.006454 29152 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.029814 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:40.013912 31260 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:40.060299 29716 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:40.064199 29125 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.065981 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:40.067505 31825 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:40.026039 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:40.022099 31821 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:40.028918 29150 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:40.025722 31773 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:40.071367 31773 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.020179 31836 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:40.075032 31836 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.089267 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.090669 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:40.091776 29716 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:40.082003 31836 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.093402 31836 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.094626 31249 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:40.094134 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 65693.2us
I20260709 13:11:40.100504 31249 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.094302 31836 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:40.107582 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:40.109448 31836 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:40.110885 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.094453 29050 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:40.112768 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.088527 31843 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.114725 31665 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:40.115085 31852 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:40.026330 31854 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:40.123888 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 128035us
I20260709 13:11:40.127235 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:40.088234 31834 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:40.090305 31452 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.129877 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.082176 31773 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.131672 29155 raft_consensus.cc:3055] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:40.137988 31431 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.088563 31823 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:40.165216 29048 leader_election.cc:336] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.780s (SENT)
I20260709 13:11:40.165518 29154 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
W20260709 13:11:40.166622 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54474.1us
I20260709 13:11:40.202454 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:40.204125 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.168371 31836 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:11:40.168830 31848 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:11:40.169979 31533 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:40.171099 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:40.171325 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:40.120908 31825 consensus_queue.cc:1243] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:40.215210 29716 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:40.016306 31824 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.121836 31435 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.124264 31025 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:40.173159 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 72026.6us
I20260709 13:11:40.060988 31844 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:11:40.069751 31821 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.145255 31348 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:40.161164 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 331598us
W20260709 13:11:40.198124 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 103748us
I20260709 13:11:40.198740 29155 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:40.149277 31665 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:11:40.164007 31847 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:11:40.144680 31455 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.164917 31854 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:40.301702 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:40.296958 29048 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.918s (SENT)
I20260709 13:11:40.176100 31863 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:40.337918 29716 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
W20260709 13:11:40.227418 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 239470us
I20260709 13:11:40.484566 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:40.507720 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 210794us
I20260709 13:11:40.380802 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:40.514367 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:40.233047 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 63748us
W20260709 13:11:40.465912 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 261858us
I20260709 13:11:40.518721 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:40.381738 31867 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:40.427241 29153 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:40.333832 29154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:40.223953 29709 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:40.196542 31352 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:40.386906 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:40.421219 31758 consensus_meta.cc:287] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.745s	user 0.010s	sys 0.000s
W20260709 13:11:40.415534 31855 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:40.523754 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:40.555572 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:40.392628 31853 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
W20260709 13:11:40.386572 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 159107us
I20260709 13:11:40.224105 31431 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:40.286029 29005 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:40.537549 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53608us
I20260709 13:11:40.521893 31679 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:11:40.538196 31844 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 13:11:40.525264 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 158522us
I20260709 13:11:40.213788 31862 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:40.227380 31249 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.313s
W20260709 13:11:40.541424 29003 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.035s	sys 0.054s
W20260709 13:11:40.574435 29003 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.035s	sys 0.054s
I20260709 13:11:40.574898 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:40.566710 31821 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:40.564786 31866 consensus_queue.cc:1175] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "befda4a6c8344c26b0f139e4e1686b3b" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:11:40.364624 31533 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:40.287814 31778 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.538393 31864 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:40.578325 31864 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.578420 29152 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:40.578713 31864 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:40.580145 31843 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:40.580302 31882 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:40.582196 31844 consensus_queue.cc:1243] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:11:40.585171 29010 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:40.224300 31823 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.588589 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:40.589294 29010 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:40.590602 31823 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.592378 31510 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:40.592763 31510 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:40.593568 31510 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.391945 31773 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.594753 31510 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:40.594533 31773 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.595041 31882 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.595556 31882 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.596661 31773 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:40.599262 28975 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.346226 31480 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:40.368069 31860 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:40.555383 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 443262us
I20260709 13:11:40.378234 31861 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.179s
I20260709 13:11:40.540649 31880 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:11:40.394217 31824 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.330309 31435 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.223s
I20260709 13:11:40.564795 29005 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:40.564569 31431 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.564905 31260 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:40.561596 31868 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:40.565392 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:40.626161 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60831.4us
W20260709 13:11:40.597820 31452 consensus_queue.cc:1175] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "243d4ae4a4ae48b2979e32874472d256" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:11:40.574278 31862 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:40.341329 31834 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.541986 29154 raft_consensus.cc:606] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:40.639427 29154 raft_consensus.cc:614] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:40.338232 31871 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:40.422870 31648 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.645359 31648 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:40.646122 31648 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:40.646739 31648 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:40.611025 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46977.8us
I20260709 13:11:40.650103 29157 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:40.611361 29152 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:40.612423 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:40.561302 31765 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.668s	user 0.011s	sys 0.000s
I20260709 13:11:40.618001 29682 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:40.598106 31882 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.525538 31836 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:40.618589 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 94942.9us
I20260709 13:11:40.622956 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:40.660154 29076 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:40.662441 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 50095.1us
I20260709 13:11:40.664140 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.665401 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.619313 31877 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:40.667893 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:40.624258 31851 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:40.673898 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:40.667521 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:40.676450 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:40.672148 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 54231.5us
I20260709 13:11:40.642068 31348 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:40.616583 31533 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:11:40.631024 31862 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:40.685071 29682 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.557512 31867 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.659643 31860 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:40.686028 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:40.692404 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:40.691002 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:40.670385 31841 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:40.700742 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 74456.1us
I20260709 13:11:40.673035 31778 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:40.672711 31767 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:40.670307 31877 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:40.708323 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58691.1us
I20260709 13:11:40.713037 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:40.684628 31870 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:40.686551 31843 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:40.686646 31873 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:40.780061 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:40.765901 29680 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:40.781417 29002 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:40.685405 31885 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:40.687394 31348 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.687139 31869 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:40.691465 31773 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:40.700337 31533 consensus_queue.cc:1243] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:11:40.807164 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 94225.5us
I20260709 13:11:40.807436 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:40.796577 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 107669us
I20260709 13:11:40.684758 31851 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:40.807969 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:40.703871 31865 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:40.766865 31879 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:40.810947 28975 leader_election.cc:400] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7.
I20260709 13:11:40.642376 31868 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:40.767242 31836 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:40.820291 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 129351us
I20260709 13:11:40.770368 31877 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:40.783274 31856 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:40.700451 31758 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
I20260709 13:11:40.780373 31843 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.780902 31765 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.826757 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:40.812134 31865 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:40.804993 31767 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:40.807080 31348 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:40.784300 31812 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.807274 31773 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:40.815611 31851 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:40.769467 31876 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:11:40.820742 31260 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:40.828962 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:40.847752 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 142596us
I20260709 13:11:40.848224 29002 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:40.849563 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:40.817312 31873 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:40.859617 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.860221 29157 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:40.860550 29682 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:40.853173 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 45266.9us
I20260709 13:11:40.826089 31823 raft_consensus.cc:2764] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:11:40.861276 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 52718.8us
I20260709 13:11:40.862377 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.712620 31841 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.838961 31843 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:40.875327 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.862142 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:40.879962 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:40.835361 31854 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:40.851339 31865 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:40.875172 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.881227 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.894538 29158 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:40.852666 31648 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:40.839234 31352 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:11:40.901007 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:40.856173 31435 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:40.852942 31773 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:40.851832 31765 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:40.864099 31866 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:11:40.875095 31639 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:11:40.906500 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:40.856353 31870 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.854933 31879 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:40.904340 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.075563 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.044323 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:41.023892 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 97363.2us
I20260709 13:11:40.921901 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:41.006958 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.093114 29002 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:41.029807 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:40.960027 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:41.018760 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:40.931924 29009 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:40.875320 31812 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.007205 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:41.099886 28975 leader_election.cc:336] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 2.038s (SENT)
I20260709 13:11:41.091342 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:40.968739 29155 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.101102 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:41.102543 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 178305us
I20260709 13:11:41.102044 31025 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:40.900542 31431 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.067520 29076 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:40.922230 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:40.906778 31885 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:41.080475 31892 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
W20260709 13:11:41.071368 31872 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:11:40.929661 31665 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:41.104537 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:41.106330 29155 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:41.096524 29005 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:41.091133 31879 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.968993 31867 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:40.910728 31821 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:41.079162 31849 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.965s	user 0.009s	sys 0.000s
I20260709 13:11:40.976792 31873 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.133929 31622 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:41.107048 31824 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:40.926155 31823 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.204s
I20260709 13:11:40.942256 31875 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:41.102648 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:41.126978 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1728) took 1134 ms. Trace:
I20260709 13:11:40.959892 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:41.128901 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 99227.6us
I20260709 13:11:40.960361 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:40.932590 31897 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:40.942495 31894 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.105150 31025 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.122434 29001 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:40.941913 31881 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:40.928488 31260 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.206s
I20260709 13:11:40.962535 31859 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.168s
W20260709 13:11:41.168282 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 65704us
I20260709 13:11:41.181392 29155 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:41.182173 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2102) took 1193 ms. Trace:
I20260709 13:11:41.182719 29155 rpcz_store.cc:276] 0709 13:11:39.988856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.131073 (+142217us) service_pool.cc:224] Handling call
0709 13:11:40.131484 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:40.968424 (+836940us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b54001a6420. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157bf06f 00007f4115837f60 00007f41158397de 00007f4115843b49 00007f4115844416 00007f4115840d8f 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62
0709 13:11:41.181873 (+213449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.182111 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123232000}
I20260709 13:11:41.129453 31665 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:41.133104 31879 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:41.168298 29005 rpcz_store.cc:276] 0709 13:11:39.992589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.992759 (+   170us) service_pool.cc:224] Handling call
0709 13:11:39.993120 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:40.228355 (+235235us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b4c00436c40. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f4113b0da20 00007f4113b1ea29 00007f4113ae4644 00007f4113b0bf2d 00007f4113b0bb93 00007f411583c18b 00007f411583ccb5 00007f4115837f30 00007f41158397de 00007f4115843b49 00007f4115844416
0709 13:11:41.126569 (+898214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.126899 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6114176}
I20260709 13:11:41.171001 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.105655 31812 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.098013 31870 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.173503 31885 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:41.142711 31824 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:41.168401 31867 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:41.171794 31025 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:41.176677 31622 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:41.200176 29125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.014s	sys 0.027s
W20260709 13:11:41.220201 29125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.015s	sys 0.027s
W20260709 13:11:41.220789 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 665207us
I20260709 13:11:41.221815 31533 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:41.173041 31875 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:41.187959 31249 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:11:41.172000 31881 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:41.206892 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.238428 29155 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
W20260709 13:11:41.193554 31510 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.003s
I20260709 13:11:41.166996 31899 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.223479 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:40.971374 31767 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.227372 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:41.228716 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.189500 31665 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:41.185104 31868 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.267704 29711 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:41.223531 31875 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:41.231419 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 126942us
I20260709 13:11:41.268934 29050 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:41.232466 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.232666 31533 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.233269 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.170331 31699 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:11:41.244751 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.225064 29716 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:41.253305 29714 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:41.208005 31824 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:41.244240 31881 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:41.257576 31870 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.168974 31897 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.169351 31894 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.266660 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.208189 31867 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:41.267313 31812 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.270539 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.277658 31529 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:41.308411 31529 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:41.308596 29155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "824fc688db264856b025a29c651eec78"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:41.309201 29155 raft_consensus.cc:606] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:41.309537 29155 raft_consensus.cc:614] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:41.309206 31529 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.312494 31529 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:41.277204 31890 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:41.316061 31890 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:41.316354 31882 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:41.316740 31882 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:41.273607 31887 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.449s	user 0.000s	sys 0.001s
I20260709 13:11:41.316861 31890 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.262686 31899 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.318157 31890 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:41.319730 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1727) took 1328 ms. Trace:
I20260709 13:11:41.320128 29003 rpcz_store.cc:276] 0709 13:11:39.991471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.991579 (+   108us) service_pool.cc:224] Handling call
0709 13:11:39.991864 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:40.574875 (+583011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:40.574907 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:40.575357 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.647530 (+ 72173us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:40.647585 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.647662 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.273047 (+625385us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:41.319443 (+ 46396us) raft_consensus.cc:1685] finished
0709 13:11:41.319463 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.319673 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":784,"raft.queue_time_us":197727,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":16999,"spinlock_wait_cycles":11354496,"thread_start_us":654359,"threads_started":2,"wal-append.queue_time_us":114698}
I20260709 13:11:41.220036 31885 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:41.268105 29711 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:41.275851 29050 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.255748 31622 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.307395 31867 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:41.299983 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.340327 31765 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.276261 31859 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:41.341405 29125 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.343035 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:41.344514 31891 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:41.345034 31891 raft_consensus.cc:697] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:41.344883 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.321566 31862 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:41.326299 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 50530.9us
I20260709 13:11:41.326759 31922 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:41.327795 31885 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.275967 31841 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.282537 31629 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:41.329955 31851 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.360466 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.361357 29004 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:41.362545 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.366990 29005 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:41.334447 31865 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.336027 29157 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:41.336938 31897 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.307734 31836 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:41.304028 29009 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
W20260709 13:11:41.306346 31864 consensus_meta.cc:287] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.003s
I20260709 13:11:41.347929 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.322458 29711 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.372319 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:41.380218 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.374763 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.381582 29712 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:41.300644 31533 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.337746 31894 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:41.384651 29711 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:41.382503 29050 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:41.354609 31765 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:41.390288 29050 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.376163 31825 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:41.378575 31862 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:41.375533 31856 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:41.337766 31843 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.345932 31891 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.336982 31882 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.385426 31922 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.385330 31910 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:11:41.392704 31348 consensus_meta.cc:287] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.002s
I20260709 13:11:41.369477 31629 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:41.402802 29153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1332) took 1621 ms (client timeout 2038 ms). Trace:
W20260709 13:11:41.415377 29153 rpcz_store.cc:269] 0709 13:11:39.774390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.917670 (+143280us) service_pool.cc:224] Handling call
0709 13:11:41.395393 (+1477723us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54002a2bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:41.395514 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1169859328}
I20260709 13:11:41.393878 31836 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:41.400925 31622 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:41.393905 31921 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:41.401072 31856 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:41.419641 29050 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.385704 31864 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.426798 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:41.396570 31765 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.396821 31773 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.431134 29050 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.434937 31867 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.429297 31921 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:41.429101 31910 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:41.436728 31859 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:41.420420 31622 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:41.405956 31665 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.413254 31844 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
W20260709 13:11:41.456941 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 78522.7us
I20260709 13:11:41.458561 29157 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:41.420997 31348 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.451337 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.412878 31025 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.416462 31922 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.471134 31867 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.471545 31851 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.472430 31921 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.491899 31843 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:41.492442 31923 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:11:41.475149 31665 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.494190 31773 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:41.494913 31877 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.000s
I20260709 13:11:41.495519 31765 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.495317 31877 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.497380 31864 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.497864 31877 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.498171 31865 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.467572 31859 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:41.499557 31879 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.500456 31859 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.501665 31879 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:41.477344 31629 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.448385 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:41.505733 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 57445.3us
I20260709 13:11:41.507545 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.509745 29080 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:41.483140 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.483189 29005 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:41.453666 31622 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:41.453011 31260 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:41.485409 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.487373 29008 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:41.490139 31025 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.493402 31922 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.513716 29080 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:41.491808 31348 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.531989 29076 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:41.515254 29081 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:41.530373 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:41.531062 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.534663 29077 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:41.535405 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.535890 29077 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:41.536520 29712 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:41.536885 29082 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:41.532464 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.537726 29080 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:41.536916 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:41.539119 29002 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:41.542943 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.543761 29081 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:41.544878 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.545529 29081 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:41.547392 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.547963 29079 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:41.523588 31910 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:41.549925 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.551216 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.555624 29076 raft_consensus.cc:2410] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:11:41.557467 29076 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:41.559664 29076 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:41.564558 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.552775 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.584442 29082 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:41.589651 29002 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:41.593087 29079 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:41.523001 31773 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:11:41.597044 29711 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:41.602486 31699 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:41.602401 31864 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:41.608409 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.611100 29159 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:11:41.615218 28976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.011s	sys 0.014s
W20260709 13:11:41.640964 28976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.011s	sys 0.014s
W20260709 13:11:41.641273 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 939160us
I20260709 13:11:41.641921 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:41.642778 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.551427 31910 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.523415 31824 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.598052 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:41.649410 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51431.8us
I20260709 13:11:41.649775 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.650094 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:41.651163 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:41.651388 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.607620 31856 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.626495 29077 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:41.660320 29125 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:41.661266 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:41.662693 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.664510 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:41.657114 31852 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:11:41.596589 31891 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:11:41.624758 29081 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:41.639004 29002 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:41.657662 31910 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:41.658586 31025 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:41.677352 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.658563 31902 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:41.672565 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.657421 29080 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:41.670977 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.693813 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.665827 31862 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:41.658650 31352 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:41.659058 31889 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:41.663622 31859 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:41.691817 28976 connection.cc:256] RPC call timeout handler was delayed by -1.8957573830000456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
I20260709 13:11:41.663884 31877 raft_consensus.cc:2764] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:11:41.694559 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:41.660965 31529 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:11:41.695544 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.711913 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.720005 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.713224 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.725456 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.725710 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.721024 29079 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:41.723541 28976 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.677268 31480 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:41.668227 31921 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:11:41.696308 31902 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:41.727034 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.730999 29004 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:41.712574 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.727411 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.728221 29080 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:41.732319 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.707679 31352 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:41.733366 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.728468 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.728881 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.722289 31825 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:41.734163 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.720181 31025 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:41.696462 31249 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:41.735700 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.736788 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:41.738651 31934 log.cc:927] Time spent T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:11:41.739941 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:41.740746 31936 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:11:41.719776 31860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.017s	sys 0.031s
W20260709 13:11:41.770845 31860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.017s	sys 0.031s
I20260709 13:11:41.740756 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:41.744741 31824 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.771495 31875 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.735419 31352 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:41.745925 31910 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.734851 31867 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:41.748373 31883 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.965s	user 0.000s	sys 0.003s
I20260709 13:11:41.749830 31533 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:41.752803 31825 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.750635 29083 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:41.785096 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.786134 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.787029 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.787644 29083 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:41.751168 31851 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:41.751514 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.789217 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:41.789757 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.790105 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.790777 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.757104 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.757027 29001 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:41.765946 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.765552 31025 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.732613 31902 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:41.767652 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.721993 31912 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:11:41.750242 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:41.783178 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.800141 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.802385 29083 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:41.775259 31867 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:41.804080 29084 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:41.788875 31825 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:41.822713 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:41.835717 29081 raft_consensus.cc:2393] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:11:41.843837 31938 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:11:41.849574 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 55440.1us
I20260709 13:11:41.797614 29709 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:41.802634 31905 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:41.855466 29683 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:41.794553 31890 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:41.848927 31629 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:41.850750 31867 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:41.863117 29084 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:41.865276 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:41.873714 31622 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.843917 31825 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.874624 31835 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:41.877274 31835 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:41.877718 31622 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.878165 31835 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.879432 31905 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:41.875223 31858 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:41.883059 31875 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.883013 31858 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.885222 31858 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:41.881410 31910 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:41.886723 31905 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:41.888760 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:41.878801 31825 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.887080 31902 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.889329 29007 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:41.889598 29001 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:41.889825 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:41.899348 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.902738 29001 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:41.906246 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1347) took 1043 ms. Trace:
I20260709 13:11:41.906666 29159 rpcz_store.cc:276] 0709 13:11:40.862633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.862736 (+   103us) service_pool.cc:224] Handling call
0709 13:11:40.894193 (+ 31457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:41.640870 (+746677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.906031 (+265161us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b54002b66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:41.906194 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1647834368}
I20260709 13:11:41.916800 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:41.925545 31860 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:41.926721 31860 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:41.927127 31860 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:41.922828 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:41.931273 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:41.931490 31480 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:41.931921 31480 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:41.932329 31480 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:41.922652 31533 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:41.941515 31533 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.941916 31533 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:41.946717 29125 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:41.947243 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 197063us
I20260709 13:11:41.948160 29009 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:41.949234 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:41.949528 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:41.950096 29004 raft_consensus.cc:2410] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 8 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:41.950273 29003 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:41.951650 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.952378 29006 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:41.952992 31857 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:41.953493 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 367) took 1058 ms. Trace:
I20260709 13:11:41.953830 29006 rpcz_store.cc:276] 0709 13:11:40.895160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.895343 (+   183us) service_pool.cc:224] Handling call
0709 13:11:40.912439 (+ 17096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:41.952751 (+1040312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.953293 (+   542us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001d79d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:41.953444 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391131904}
I20260709 13:11:41.955163 29079 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:41.956509 29002 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:41.952497 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.950215 31877 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:41.960389 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.962651 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.962890 29007 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:41.964375 29078 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:11:41.965476 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.966254 29001 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:41.967295 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.968544 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.961869 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:41.969228 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.969442 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.969731 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:41.969849 29154 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:41.969399 29003 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:41.970494 29157 raft_consensus.cc:2368] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:41.971367 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.971830 29716 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:41.973209 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.974985 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.977376 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:41.979383 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:41.980661 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.981419 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:41.981956 29158 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:41.982197 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.982208 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:41.984871 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.990233 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.991595 29048 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:41.992425 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:41.996670 29154 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:41.997627 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.991782 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.998754 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:41.999707 29155 raft_consensus.cc:2410] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:11:41.997246 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:42.002481 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:41.978859 31864 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.009313 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.010046 29157 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:41.976444 31858 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.977891 31824 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:41.976382 31859 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:41.968135 31861 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:41.976743 31877 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:42.033298 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:41.980450 31922 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:42.044879 31922 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:41.980451 31891 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:41.997200 31902 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:42.041075 31858 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.048229 31858 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:41.998020 31699 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:41.998958 31889 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:42.049173 31889 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:42.048730 31877 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.049993 31889 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.051396 29003 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:41.997872 31885 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:42.042084 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:42.042450 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.043001 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:42.056912 29155 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:42.007400 31864 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:41.997589 31773 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:42.038971 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:42.054173 31950 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:11:42.063323 29080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:42002
I20260709 13:11:42.073062 29080 raft_consensus.cc:606] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:42.073402 29080 raft_consensus.cc:614] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:41.957326 31860 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.075384 31860 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:42.074384 31352 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.058254 31885 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.095961 29154 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
W20260709 13:11:42.070791 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 88131.7us
I20260709 13:11:42.099750 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.106004 29155 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:42.106055 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:42.099225 29158 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:42.107506 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.108098 29158 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.108345 28976 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:42.110219 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:42.115597 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.117367 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:42.117848 29153 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:42.118207 29154 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:42.119840 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.120546 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:42.121189 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:42.121584 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:42.122313 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:42.122879 29158 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:42.123256 29154 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:42.123688 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:42.124388 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.131019 29154 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:42.126451 29003 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:42.096468 29157 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:42.084501 31352 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:42.099184 31881 consensus_meta.cc:287] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.002s
W20260709 13:11:42.128470 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 136963us
I20260709 13:11:42.114342 31435 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:42.152524 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:42.083648 31480 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.084604 31885 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:42.129896 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 151027us
I20260709 13:11:42.125427 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:42.055681 31861 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.132141 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.133162 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:42.133775 29153 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:42.145128 29154 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:42.162570 29154 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:11:42.162508 29050 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:42.148864 29155 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:42.154244 28975 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:42.131232 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:42.065337 31867 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.165161 29155 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:42.157720 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.172227 29158 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.178438 29158 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:42.181186 29050 leader_election.cc:400] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:42.181649 29050 leader_election.cc:403] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:11:42.132241 31765 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:42.184863 29153 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
W20260709 13:11:42.184238 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 62718.4us
W20260709 13:11:42.208603 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.215131 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:42.216054 29005 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:42.158696 31260 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:42.188520 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.207774 29152 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:42.154590 31851 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:42.188866 29150 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:42.150233 31857 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:11:42.229516 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:42.187351 31879 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:42.235320 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:42.232278 29007 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:42.212546 31910 raft_consensus.cc:2764] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:11:42.236991 29050 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:42.214435 31773 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:42.227536 31622 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.241106 31864 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:42.240923 31622 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.243003 31622 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:42.228101 31765 raft_consensus.cc:2764] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:11:42.243531 31861 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:42.243902 31765 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:42.244359 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:42.244796 31765 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:42.245414 31765 consensus_queue.cc:1243] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:11:42.246026 31867 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:42.236579 31879 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.250169 31879 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:42.158999 31881 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.253852 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:42.185730 31956 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:42.245872 31765 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:42.181092 31533 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.261353 29008 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:11:42.278729 29008 pending_rounds.cc:85] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 2
I20260709 13:11:42.279062 29008 pending_rounds.cc:107] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:11:42.279304 29008 raft_consensus.cc:2889] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:42.264889 29712 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:42.270123 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:42.266434 31480 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:42.234632 31699 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.225234 31851 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.270337 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:42.292356 29009 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.273313 29005 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:42.296664 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:42.297792 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:42.275326 29152 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:42.257668 31260 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:42.271101 31867 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:11:42.285396 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 102044us
I20260709 13:11:42.268102 31956 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:42.287678 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:42.288726 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:42.301537 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:42.226943 31865 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:11:42.302837 29152 raft_consensus.cc:606] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:42.303102 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:42.304698 29159 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:42.312789 29708 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:42.230182 31905 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:11:42.318282 29152 raft_consensus.cc:614] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:42.295919 31851 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:42.291262 31859 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:42.281672 31957 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:42.295764 31881 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.308706 31960 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:42.341408 28975 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:42.312892 31773 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:42.304412 31824 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:42.307292 31966 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:42.295439 31699 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.310518 31867 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:42.314349 31956 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:42.365396 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:42.374014 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 72559.5us
I20260709 13:11:42.384419 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:42.386945 29009 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:42.335772 31260 consensus_queue.cc:1059] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:11:42.402138 31957 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:42.409780 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 98873.4us
I20260709 13:11:42.349483 31966 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:42.354254 31955 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:11:42.424917 31910 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:42.430989 31533 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:42.405157 31824 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:42.408185 31947 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:42.446089 29152 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:42.454911 29152 raft_consensus.cc:606] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:42.405725 31765 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:42.414467 31957 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.465153 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:11:42.465085 31765 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.483613 31765 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:42.484305 31967 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:11:42.484479 31921 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:42.485193 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.436518 31260 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:11:42.487022 31973 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:11:42.490828 31845 raft_consensus.cc:3055] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:42.491197 31845 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:42.491983 31845 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.493237 31845 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:42.494308 31952 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.272s	user 0.000s	sys 0.000s
I20260709 13:11:42.496377 31922 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:42.496778 31922 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:42.497573 31922 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.498747 31966 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.500155 31966 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.500383 31899 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:42.481513 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.469313 31824 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:42.483003 31959 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:11:42.483163 31910 raft_consensus.cc:3055] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:42.503100 31910 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:42.503990 31910 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.457634 31885 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.453050 31867 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.502727 31882 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:42.505156 31910 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:42.507432 31921 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:42.469928 31957 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.514073 31921 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:42.505748 31955 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:42.453562 31965 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:42.515775 31965 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.516175 31965 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.523102 29155 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:11:42.526417 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:42.542790 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 57664.8us
I20260709 13:11:42.543831 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:42.550112 31960 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:42.551354 31960 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:42.555958 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 138457us
I20260709 13:11:42.557835 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:42.564657 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.566309 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.571346 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.572659 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.577869 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:42.581063 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.582459 29005 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:42.545994 31866 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.584103 29005 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:42.545200 31455 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:42.585590 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 201258us
I20260709 13:11:42.577082 31955 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:42.563568 31856 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.495796 31956 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.588095 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.594349 29002 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:42.576897 31873 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:42.589097 31699 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:42.579408 31876 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:42.600466 31856 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:42.544835 31851 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.584378 31877 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:42.602121 28975 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.602505 31851 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.602898 31877 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:42.604712 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:42.606011 31877 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:42.607187 28975 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:42.607968 31877 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:42.608582 31877 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:42.609434 31877 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.593009 31955 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:42.612936 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.613864 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:42.592978 31981 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:42.586194 31866 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.611537 31879 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.628804 29159 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:42.632550 31963 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:42.635612 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.636368 29155 raft_consensus.cc:2368] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:42.636904 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.637645 29152 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
W20260709 13:11:42.638423 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.639830 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:42.640352 29002 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:42.620790 31885 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.622905 31955 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:42.642665 28975 leader_election.cc:400] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:42.623167 31851 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:42.645462 29159 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:42.645701 28975 leader_election.cc:403] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:11:42.641516 31860 consensus_meta.cc:287] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.004s
I20260709 13:11:42.649341 29152 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:11:42.651177 29152 pending_rounds.cc:85] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 3
I20260709 13:11:42.651423 29152 pending_rounds.cc:107] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:11:42.651643 29152 raft_consensus.cc:2889] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:42.651844 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:42.652575 29159 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:42.629459 31773 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:42.655970 31974 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:11:42.657733 31851 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:42.658763 31876 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:42.727494 29002 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:42.655988 31979 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:42.660871 31963 consensus_queue.cc:1059] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:42.649705 31969 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:42.684360 31956 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.692776 31885 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.693053 31879 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.697137 31866 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.628065 31856 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:11:42.721312 31957 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.744843 31965 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.748570 31835 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:11:42.748234 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.745051 31851 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.747570 31955 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.770330 29155 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
W20260709 13:11:42.746690 31964 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.289s	user 0.003s	sys 0.000s
I20260709 13:11:42.771291 29152 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:11:42.772154 31773 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.766263 31765 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.777395 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:42.778466 29008 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:42.781070 31974 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:42.783812 31922 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.791882 29010 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:11:42.775812 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:42.834283 29007 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:42.809264 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:42.835178 29003 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:42.810698 29151 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:42.811115 29004 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:42.771528 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:42.747386 31876 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:42.823963 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:42.837764 29712 raft_consensus.cc:2368] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:42.817492 31767 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.838573 31767 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.839123 31957 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.825402 31885 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.842224 28976 leader_election.cc:400] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:11:42.824788 31843 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:11:42.850399 29080 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:42.826377 31835 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:42.830856 29005 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:42.847955 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:42.843489 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 72040.5us
I20260709 13:11:42.863001 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:42.841308 31876 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:42.866042 29001 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:11:42.867568 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:42.868326 29085 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:42.869357 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:42.832405 31981 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:42.873121 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:42.840652 31974 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:42.878039 31876 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.880322 29079 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:42.860090 31882 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:42.865851 31957 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:42.867158 31455 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:42.904978 29085 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:11:42.918995 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:42.919579 29079 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:42.920181 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:42.918674 29010 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:42.922668 29085 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:42.925355 29085 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:42.871656 31249 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:11:42.912179 31985 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:42.937420 29085 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:42.927585 31876 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.911988 31909 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:11:42.915972 31845 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:11:42.923273 31882 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:42.968099 31882 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:42.969460 29151 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1342) took 2110 ms (client timeout 2113 ms). Trace:
W20260709 13:11:42.969894 29151 rpcz_store.cc:269] 0709 13:11:40.859540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.859923 (+   383us) service_pool.cc:224] Handling call
0709 13:11:42.969282 (+2109359us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54002b57d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:42.969402 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3914904832}
I20260709 13:11:42.933117 31835 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.972920 31835 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:42.973106 31989 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:11:42.973796 29682 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:42.936646 31985 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:42.976570 31991 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:11:42.977697 31985 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:42.931181 31974 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:42.978567 31908 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:42.979205 31974 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.980621 31955 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:42.918884 31957 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.984875 31957 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:42.928478 31455 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:42.988592 31899 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:42.988483 31455 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.989172 31899 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:42.990078 31899 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:42.991039 31899 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:42.992252 28975 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 2.113s (SENT)
I20260709 13:11:42.992681 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:42.965662 31966 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:43.000149 29084 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
W20260709 13:11:43.000919 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:43.002301 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.988680 31765 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:43.005630 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:43.015597 29009 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:43.030959 31767 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:43.027691 31985 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:11:42.996097 31966 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:43.001817 31955 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:43.040027 31980 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:43.040416 31965 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:43.050637 29078 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
W20260709 13:11:43.081445 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.045768 31960 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:43.048724 31980 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.094489 29708 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:43.073191 31843 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.102351 31480 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:43.123320 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2110) took 1150 ms. Trace:
I20260709 13:11:43.123725 29150 rpcz_store.cc:276] 0709 13:11:41.973313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.973423 (+   110us) service_pool.cc:224] Handling call
0709 13:11:43.123147 (+1149724us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54002b52d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:43.123268 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432550144}
I20260709 13:11:43.106608 31880 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:43.104489 31885 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:43.116263 31773 consensus_queue.cc:1059] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:11:43.113775 31844 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.133584 31844 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.057708 31966 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.122632 31480 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.135938 31844 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:43.138594 31966 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:43.138876 31480 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:43.144510 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.154335 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:43.155037 29080 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:43.157382 29125 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:43.095916 31960 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.158809 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:43.160144 31844 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:43.160392 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:43.160585 31844 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:43.161059 31844 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:43.156998 31480 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.163287 31480 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.164927 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:43.166159 29125 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:43.166489 31990 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.215s	user 0.001s	sys 0.002s
I20260709 13:11:43.166899 31699 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:43.161703 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:43.170436 29080 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:43.172888 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:43.173189 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:43.175503 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.114974 31843 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:43.136452 31992 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:11:43.198045 31954 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.930s	user 0.003s	sys 0.001s
I20260709 13:11:43.212852 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:43.108573 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 134836us
I20260709 13:11:43.198972 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:43.196722 31909 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:43.375978 29079 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:11:43.291771 31533 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:43.299404 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:43.289775 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:43.301101 29711 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:43.300060 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:11:43.290585 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 43981.8us
I20260709 13:11:43.289460 31843 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:43.304275 29710 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:43.284291 31868 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:43.298564 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:43.155704 31880 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.299283 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:43.326359 31960 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:43.300351 29716 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:43.409494 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 119811us
I20260709 13:11:43.349990 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:43.322469 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:43.319015 31823 raft_consensus.cc:3055] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:43.351405 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:43.352782 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 77597.4us
I20260709 13:11:43.415756 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:43.353013 29709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 618) took 2394 ms. Trace:
I20260709 13:11:43.419828 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:43.420599 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:43.422367 29084 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:11:43.376145 31909 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:43.318116 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:43.429523 29085 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:43.283363 31773 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:43.438757 31773 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.284412 31844 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.290349 29050 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:43.419878 29709 rpcz_store.cc:276] 0709 13:11:40.958464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.958635 (+   171us) service_pool.cc:224] Handling call
0709 13:11:40.958998 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:41.860033 (+901035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.352154 (+1492121us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54004b69d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:43.352386 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1872372992}
I20260709 13:11:43.419624 31868 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.446077 31868 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:43.360237 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 161315us
I20260709 13:11:43.363827 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:43.362524 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:43.377077 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:43.389582 29715 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:43.385682 29683 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:43.449715 29079 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:43.379339 31922 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.001s
I20260709 13:11:43.410141 29682 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:43.290657 31894 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:43.378767 31871 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:43.426510 29717 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:43.427363 28975 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:43.429021 29080 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:11:43.433609 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 344) took 1446 ms. Trace:
I20260709 13:11:43.433769 29083 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:43.353449 31899 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 13:11:43.434739 32000 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:11:43.401270 31533 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:43.435966 31996 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:11:43.448710 31909 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:43.437314 31995 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:11:43.438299 31993 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.471s	user 0.001s	sys 0.009s
I20260709 13:11:43.357350 31968 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:43.449955 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 64404.8us
I20260709 13:11:43.494581 29152 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:43.557740 29081 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:43.507222 29079 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:43.357774 31966 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:11:43.474891 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:43.477819 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 67757us
I20260709 13:11:43.477079 29717 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:43.479169 29151 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:43.565083 29682 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:43.480185 29159 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:43.479023 29154 rpcz_store.cc:276] 0709 13:11:41.986806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.163314 (+176508us) service_pool.cc:224] Handling call
0709 13:11:43.123299 (+959985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.123557 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.123598 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.123645 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:43.123712 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.123797 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.433439 (+309642us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54002b52d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:43.433550 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2206920064}
I20260709 13:11:43.481748 29153 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:43.415094 31823 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:43.484759 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 57485.5us
I20260709 13:11:43.430994 31843 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:43.436340 31965 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:43.567293 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:43.488405 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:43.418953 31455 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:11:43.481428 31957 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:43.560178 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:43.451941 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 59278.3us
I20260709 13:11:43.451771 31845 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:43.568387 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:43.568531 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:43.570367 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:43.601457 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 311153us
I20260709 13:11:43.616961 29155 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:43.603394 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:43.616091 29154 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:43.620586 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60491.4us
I20260709 13:11:43.623113 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:43.623714 29152 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:43.484157 31835 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:43.602427 29717 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.632185 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:43.621752 29153 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:43.636345 29153 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:11:43.636869 29153 pending_rounds.cc:85] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 3
I20260709 13:11:43.637223 29153 pending_rounds.cc:107] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:11:43.637596 29153 raft_consensus.cc:2889] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:43.493878 31908 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:11:43.640162 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:43.649475 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:43.488874 31876 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:43.485869 31894 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.576498 31249 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.652407 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:43.653126 29151 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:43.660081 29717 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:43.653578 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.570101 31968 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.575526 31922 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:43.560675 31974 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:11:43.573973 32007 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:43.600819 31860 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:43.661717 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2118) took 1685 ms. Trace:
I20260709 13:11:43.689149 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:43.689350 29157 rpcz_store.cc:276] 0709 13:11:41.976489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.160876 (+184387us) service_pool.cc:224] Handling call
0709 13:11:42.161254 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:42.165850 (+  4596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:42.165886 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:42.758774 (+592888us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b2c0012c0f8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:42.759251 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:11:42.880241 (+120990us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:42.880281 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:42.880354 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:43.344939 (+464585us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:43.661449 (+316510us) raft_consensus.cc:1685] finished
0709 13:11:43.661478 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.661647 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4197,"raft.queue_time_us":5472,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":4118,"spinlock_wait_cycles":103960704,"thread_start_us":120564,"threads_started":1,"wal-append.queue_time_us":207331}
I20260709 13:11:43.618862 31843 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:43.682729 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:43.682842 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:43.685423 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 120425us
I20260709 13:11:43.640272 31844 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:43.639879 31981 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:43.692020 29157 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:43.692656 29150 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:43.698103 29002 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:11:43.665781 31845 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:43.648151 32004 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:43.669718 31955 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:43.704952 29002 pending_rounds.cc:85] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:11:43.699383 29153 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:43.663275 31455 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:43.693100 29682 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:43.680269 32014 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.782203 29002 pending_rounds.cc:107] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:11:43.667364 31835 raft_consensus.cc:697] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:43.824707 29717 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:11:43.829293 29002 raft_consensus.cc:2889] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:43.831086 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:43.622143 31957 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:43.807278 29152 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
W20260709 13:11:43.854346 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 160560us
I20260709 13:11:43.771009 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:43.856648 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:43.832512 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:43.832711 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 385590us
I20260709 13:11:43.672191 31249 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.800129 29154 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:43.859527 29151 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:43.779709 29150 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:43.863550 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1365) took 1702 ms. Trace:
W20260709 13:11:43.762856 32006 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.348s	user 0.004s	sys 0.000s
W20260709 13:11:43.742295 32001 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:11:43.684676 31922 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.864004 29158 rpcz_store.cc:276] 0709 13:11:42.160902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.179119 (+ 18217us) service_pool.cc:224] Handling call
0709 13:11:42.179406 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:42.180088 (+   682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:42.180128 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:43.242303 (+1062175us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b2c0012c0f8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:43.275924 (+ 33621us) log.cc:844] Serialized 26 byte log entry
0709 13:11:43.578692 (+302768us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:43.578757 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.582057 (+  3300us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:43.853460 (+271403us) raft_consensus.cc:1685] finished
0709 13:11:43.853479 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.863489 (+ 10010us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1701,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":4368,"spinlock_wait_cycles":2653568,"thread_start_us":248675,"threads_started":1,"wal-append.queue_time_us":356589}
I20260709 13:11:43.823695 31856 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.698752 32007 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.720261 31871 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:43.868084 31880 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:43.869267 31968 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:43.869966 29157 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:11:43.722543 31533 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:43.870837 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.870988 29152 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:43.870741 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:43.832782 31955 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:43.832078 31960 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:43.825269 31965 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:43.741468 32003 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:11:43.702489 31859 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.165s
I20260709 13:11:43.745002 32012 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:43.703619 31823 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.857817 31957 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:43.830116 32014 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.920576 31955 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:43.887737 29159 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:43.863705 31999 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:43.754580 31773 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:43.824360 32008 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.149s
I20260709 13:11:43.869729 31249 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:43.893693 31985 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:43.960546 31985 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:43.961414 31985 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.851115 31455 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.965915 31455 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:43.853722 31966 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:11:43.968273 32014 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:43.968209 31966 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:43.887778 31856 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.713500 31981 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.743847 32004 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.888818 31871 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:43.924624 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 303271us
I20260709 13:11:43.932570 32012 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:43.945271 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.958729 31999 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:43.917156 29716 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:11:43.973388 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.920097 31968 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:11:43.886732 31835 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.969846 31921 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:43.990067 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 412) took 1140 ms. Trace:
I20260709 13:11:44.006786 29077 rpcz_store.cc:276] 0709 13:11:42.849188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.849315 (+   127us) service_pool.cc:224] Handling call
0709 13:11:42.875619 (+ 26304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:42.996868 (+121249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:42.996898 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:42.996934 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:42.996963 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.989796 (+992833us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54000244d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:43.989854 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.989997 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278435712}
I20260709 13:11:43.924309 31965 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:43.937345 31824 consensus_queue.cc:1059] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:11:43.924144 31960 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:43.976316 31533 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.994068 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:44.062247 29708 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:44.057899 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:44.058074 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 208022us
I20260709 13:11:44.063555 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:44.070453 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 213876us
I20260709 13:11:43.991611 32012 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:44.072803 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 200426us
I20260709 13:11:44.073130 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.077504 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:44.006686 31921 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:44.066156 31894 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:44.076409 31876 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:44.071496 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:44.070588 32021 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.091255 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:44.099462 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:44.105530 31867 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:44.084106 31845 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:44.113529 31025 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:44.116068 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:44.069669 31533 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.120836 29077 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:44.121304 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:44.087215 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:44.099558 32018 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:44.089644 31897 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.085080 31529 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:11:44.093385 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:44.151947 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54500.3us
I20260709 13:11:44.088017 31880 consensus_queue.cc:1059] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:11:44.119845 31859 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:44.171419 29077 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
W20260709 13:11:44.092301 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:44.145467 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 80459.7us
I20260709 13:11:44.154167 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:44.156972 32022 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:44.135145 32007 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:44.164484 31894 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:44.098660 32020 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:11:44.174073 31844 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:44.212072 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 145393us
I20260709 13:11:44.146369 32018 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:44.074049 31871 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.121174 31025 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:44.112833 31876 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.237248 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.120012 29709 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:44.246270 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:44.241015 31992 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:11:44.240731 32022 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:44.275463 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:44.276439 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:44.273239 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 50113.4us
I20260709 13:11:44.277707 29080 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:44.279302 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:44.253470 32018 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:44.155081 31897 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.247363 31859 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:44.115280 32021 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.289964 29078 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:11:44.116604 31999 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:44.297432 32026 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 13:11:44.299028 32035 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:11:44.299742 29156 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:44.300583 32038 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:11:44.300519 31025 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.301611 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1652) took 1214 ms. Trace:
I20260709 13:11:44.301868 31025 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:44.268770 31985 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.302033 29085 rpcz_store.cc:276] 0709 13:11:43.087497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.430127 (+342630us) service_pool.cc:224] Handling call
0709 13:11:43.430379 (+   252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:43.449549 (+ 19170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.449580 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:43.749272 (+299692us) spinlock_profiling.cc:243] Waited 963 us on lock 0x7b2c0004c3c8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:11:43.749774 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:11:43.976825 (+227051us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:43.976857 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.976917 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:44.301279 (+324362us) raft_consensus.cc:1685] finished
0709 13:11:44.301301 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:44.301548 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1687,"raft.queue_time_us":18960,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":30168,"spinlock_wait_cycles":2213760,"thread_start_us":226636,"threads_started":1,"wal-append.queue_time_us":360096}
I20260709 13:11:44.271271 31871 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:44.302314 32025 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.274783 31773 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:11:44.312137 31981 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.332415 31965 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.177141 31921 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.289520 31629 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
W20260709 13:11:44.352491 32033 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:11:44.364848 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 56270.4us
I20260709 13:11:44.306591 32028 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:11:44.354851 32043 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.339843 31985 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.360954 32029 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.359956 31960 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.367250 32023 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.343477 32025 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.377411 32042 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:44.384135 32037 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:11:44.393184 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 239093us
I20260709 13:11:44.383841 31533 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:44.383440 31965 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:44.382004 31773 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:44.395576 29125 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:44.381553 32044 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:11:44.401268 32042 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.403901 32012 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.418642 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:44.446174 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:44.446656 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.364092 32022 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.395027 32023 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.458316 31856 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.459297 32023 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.445806 31533 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:44.462574 31965 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:44.462915 31533 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:44.463505 31533 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:44.463634 31965 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.460865 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1795) took 1084 ms. Trace:
I20260709 13:11:44.467741 29002 rpcz_store.cc:276] 0709 13:11:43.376141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.377549 (+  1408us) service_pool.cc:224] Handling call
0709 13:11:43.415290 (+ 37741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.870575 (+455285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.460617 (+590042us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54001caf54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:44.460803 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649554304}
W20260709 13:11:44.423007 31909 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.855s	user 0.006s	sys 0.006s
I20260709 13:11:44.461839 32025 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:44.471539 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.408085 31773 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:44.490477 29152 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:44.466878 32012 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.385978 31859 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.450124 31960 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:44.514683 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 270539us
I20260709 13:11:44.467415 31968 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.384276 32043 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.488232 32054 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:44.489003 32046 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:44.511888 31957 consensus_meta.cc:287] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.001s
I20260709 13:11:44.388024 32029 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.509523 32038 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.539223 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:44.514194 31867 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.515821 32014 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.525805 31856 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.560561 29717 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:44.562541 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:44.562541 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 67658.9us
I20260709 13:11:44.533879 32023 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:44.567102 29156 raft_consensus.cc:1813] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:44.479691 31909 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.538722 32051 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.534454 32007 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:44.559576 31897 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.539188 32012 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.541870 31876 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.533506 31249 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:11:44.589998 29156 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 4: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:44.536820 32049 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:44.591957 32058 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.611524 32058 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.613798 32058 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.614332 32058 consensus_queue.cc:1059] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:44.615124 32058 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.536098 32050 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:11:44.594426 32043 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.618208 32043 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.594781 32051 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:44.596110 32056 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:11:44.598413 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2131) took 1084 ms. Trace:
I20260709 13:11:44.620383 29159 rpcz_store.cc:276] 0709 13:11:43.514342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.567000 (+ 52658us) service_pool.cc:224] Handling call
0709 13:11:44.598223 (+1031223us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400139f54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:44.598344 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721408896}
I20260709 13:11:44.557855 31973 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.536795 31968 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.618678 32043 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.537400 31960 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.562685 32048 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.623587 29150 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:44.605686 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.633793 29154 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:44.625408 32012 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.634825 32064 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:44.635425 32064 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.544078 31957 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.615554 32058 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.591388 32021 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.565799 31867 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.568712 32014 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.580183 31856 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.584399 31773 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.645179 31985 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.637104 32049 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:44.565914 32038 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.693058 32049 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.694171 32038 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.543573 32046 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.696144 32046 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.696233 31909 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:44.588459 32023 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.697110 32068 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:44.542871 32054 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.690544 32051 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.628654 31876 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.629246 31973 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.679183 31922 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:44.696311 32052 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.228s	user 0.001s	sys 0.008s
I20260709 13:11:44.712386 31973 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.639804 32048 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.676714 31249 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.697515 32068 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:44.691655 32042 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.710340 32058 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.711349 32043 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.693881 31533 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.658047 31867 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.714258 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.729581 29159 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:44.730427 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.714623 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.658859 32045 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.716223 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.716696 32029 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.710381 32023 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.717115 32054 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.719615 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.726127 29715 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:44.713636 32012 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.733736 29152 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:44.747051 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.735941 32068 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.738993 31876 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.740447 32014 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.748642 32068 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:44.750730 31910 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:44.751132 31910 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:44.752770 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.741616 31856 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.742246 31960 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.717607 32064 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.747886 29154 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:44.758546 31533 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:44.721582 31249 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.760484 31980 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.761461 31249 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:44.746590 31955 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.774s	user 0.012s	sys 0.001s
I20260709 13:11:44.762434 31955 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:44.763558 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:44.763999 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:44.734808 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 172764us
I20260709 13:11:44.765436 29084 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:44.766398 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.767041 29153 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:44.768687 29084 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:11:44.768903 29078 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:44.747450 31529 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:44.772979 31957 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.745645 32045 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.775395 32045 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:44.770552 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.766968 32048 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.743458 32018 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.782658 32018 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.776226 31999 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.783880 31999 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.786047 31999 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.788501 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.797020 32004 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.797497 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:44.799003 31981 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:44.799382 31981 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:44.799275 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:44.801357 32036 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:44.802712 31955 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.803653 32046 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:44.806035 31981 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.807695 31908 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:44.818718 31767 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:44.816313 32022 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:44.823174 31981 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:44.809861 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.824477 29153 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:44.811457 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:44.825704 29154 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:44.751931 31910 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.807725 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2129) took 1294 ms. Trace:
I20260709 13:11:44.808457 32053 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:11:44.829996 29155 rpcz_store.cc:276] 0709 13:11:43.513754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.639222 (+125468us) service_pool.cc:224] Handling call
0709 13:11:43.666689 (+ 27467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.002493 (+335804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.002537 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.002592 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:44.002634 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.807390 (+804756us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54002a35d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:44.807470 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:44.807646 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771763584}
I20260709 13:11:44.832824 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:44.834431 29155 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:44.836270 31767 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:44.836370 32036 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:44.837198 31767 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.837744 32036 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.843425 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.844494 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.845000 29152 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:44.845970 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:44.839944 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.852432 29153 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:44.857924 29154 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:11:44.867375 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:44.868072 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:44.869727 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:44.876832 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:44.882464 29084 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:44.938458 31980 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:44.945052 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:44.945137 31980 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.957983 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.945922 29076 raft_consensus.cc:2410] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:11:44.951171 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 118415us
I20260709 13:11:44.963706 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.877344 31910 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:44.952246 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.966081 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.964090 29155 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:44.968932 29156 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:44.969527 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.966795 29153 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:44.967263 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:44.971210 29152 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:44.971710 29009 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:44.972466 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:44.879720 32023 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:44.974704 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:44.901765 31856 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:44.989853 31856 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.940551 31871 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.975533 29009 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:44.853430 31957 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:44.992331 31871 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:44.992959 31957 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:44.976512 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:44.978603 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:44.994603 31871 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:44.979204 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:44.980151 31960 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:44.972826 31025 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:11:44.938913 31955 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:44.995008 29007 raft_consensus.cc:2410] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:11:44.995715 29078 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:44.938982 32054 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:44.995709 29002 raft_consensus.cc:2410] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:44.997058 29156 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:44.998152 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:44.998930 29009 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:44.955339 31980 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.991817 29152 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:45.009392 31981 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.008232 29004 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:45.039516 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:44.951722 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:44.995092 31957 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:44.998976 29010 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:45.008175 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:44.976190 32078 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:45.012699 31955 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.010366 31960 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.032773 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.048733 29001 raft_consensus.cc:2410] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:11:45.050000 31960 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.051260 31955 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.052536 29006 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:45.037299 31856 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:45.050921 31981 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.057966 31981 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.036712 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.043247 32078 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.029505 32054 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.083556 32054 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:45.085794 32078 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.088338 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.089092 29077 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:45.093396 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.094050 29010 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:45.097520 29051 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:45.104092 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:45.080123 31882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.109s	user 0.005s	sys 0.004s
I20260709 13:11:45.117686 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:45.119580 31957 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:45.120131 31957 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:45.120581 31957 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:45.121937 31823 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:45.104061 32078 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:45.117882 31882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.147s	user 0.006s	sys 0.005s
I20260709 13:11:45.103850 31981 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:11:45.136612 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:45.140363 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.126538 31856 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:45.145733 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.107903 31960 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:11:45.145581 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:45.143499 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.146881 29152 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:45.139113 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:45.145738 31980 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:11:45.148434 32046 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:45.152738 29159 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:45.153993 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:45.157908 29050 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:45.163748 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 68695.2us
W20260709 13:11:45.171172 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 74204.4us
I20260709 13:11:45.171587 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.156236 31974 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:45.155726 31767 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:45.180109 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:45.162720 32014 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:45.185278 32064 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.185849 32054 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:45.168385 32029 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:45.156580 31856 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.177237 32022 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:45.199790 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:45.200207 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.205929 29159 raft_consensus.cc:2368] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
W20260709 13:11:45.205480 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 67525.2us
I20260709 13:11:45.177034 31876 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:11:45.209339 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.212503 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:45.178592 31974 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:45.202524 31955 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:45.201094 32014 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.221493 28975 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:45.209091 32022 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.229584 29050 leader_election.cc:400] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:11:45.230098 29050 leader_election.cc:403] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:11:45.230278 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.229028 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.231918 29077 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:45.213014 31876 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.234005 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:45.236640 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.214092 32029 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:45.238770 29082 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:45.202312 31767 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.204174 32064 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.222205 31974 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:45.222373 31955 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.233928 32022 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:45.242708 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.205106 32054 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:45.268507 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.300899 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.224656 32014 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:45.220880 32066 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:45.224701 32036 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:11:45.303339 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 69400.9us
I20260709 13:11:45.310921 29076 raft_consensus.cc:1813] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:45.315187 29076 raft_consensus.cc:2410] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:11:45.310279 29084 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:45.312904 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.316848 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.317371 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.239414 31629 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:45.240092 31981 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:11:45.333702 29155 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:45.333851 29153 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:45.238298 31876 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:45.332768 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.240942 31960 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.314867 31966 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:45.242837 32043 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:11:45.379156 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.327034 31629 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.310619 31955 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:45.313509 32066 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.321038 32036 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.356594 29155 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
W20260709 13:11:45.384208 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 171844us
I20260709 13:11:45.346947 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:45.355614 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.395272 29077 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:45.338441 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:45.340560 32088 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:11:45.395995 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:45.400182 29084 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
W20260709 13:11:45.337949 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.413568 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.351444 31980 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.421550 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.425827 29084 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:45.428539 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.429224 29081 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:45.429298 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.386472 31844 raft_consensus.cc:3055] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:45.353950 32054 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.435999 29155 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:11:45.441618 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.446189 29077 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:45.442028 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.448414 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.448581 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.449290 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.388319 31629 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:45.442574 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:45.342317 32076 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.106s
I20260709 13:11:45.342005 32029 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.433347 31844 raft_consensus.cc:740] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:45.453509 31910 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:45.418839 31966 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:45.454270 31844 consensus_queue.cc:260] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.455451 31844 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:45.455341 31966 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.435837 31974 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.458467 31974 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.459059 31974 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:45.459913 31965 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.460466 31965 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.460803 29085 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:45.462486 31965 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.398233 32036 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:45.341557 31957 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.465729 31957 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.450742 31980 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.449857 29008 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:45.398543 32066 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:45.469498 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.470538 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.471098 29008 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:45.455770 32078 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:45.471988 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.459806 31974 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:45.431998 31773 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:45.481427 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.482076 29008 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:45.484333 29008 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:45.380502 31960 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.437618 31767 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.490568 29076 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:45.490878 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:45.486531 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 275429us
I20260709 13:11:45.496873 31974 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:45.499671 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 76537us
I20260709 13:11:45.480585 31844 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.526057 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.557263 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.558110 29153 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:45.558391 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.503924 31767 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
W20260709 13:11:45.568280 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 72425.1us
I20260709 13:11:45.529449 31844 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:45.572544 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 146552us
I20260709 13:11:45.530017 32054 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:11:45.570555 29155 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:45.568765 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:45.541290 31965 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:11:45.581405 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:45.585080 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.585731 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:45.587787 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.588007 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.588577 29708 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:45.590204 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.569037 32104 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.563257 32098 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:45.593194 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.594918 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.562661 32076 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:11:45.553869 32022 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.597270 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:45.579408 32045 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:45.566653 32046 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:11:45.587455 32054 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.602701 31480 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:45.612097 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:45.600343 31025 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:45.610848 28976 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:45.627743 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:45.628541 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.595628 32098 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.623587 29006 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:45.600651 32043 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:45.631423 32043 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:45.626849 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.609400 32045 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.631224 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:45.615049 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 166409us
I20260709 13:11:45.610145 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.623342 31824 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:45.636400 32045 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:45.636580 32043 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.637529 29048 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:45.614943 31767 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:45.632282 32098 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:45.599906 32104 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.626920 31025 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:45.656463 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.652832 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:45.634979 31965 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:45.622507 32054 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.647080 31767 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:45.651039 32111 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:45.658946 32076 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:45.659180 31025 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:45.656358 31960 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.664139 32102 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.651118 31824 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.662313 32043 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:45.665817 32094 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:11:45.671525 32089 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.675460 32111 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.694039 32111 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.680704 32104 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:45.681807 32101 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:11:45.698640 31824 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.681981 31629 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.683866 31974 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:45.701892 32066 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.669157 31767 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:45.689702 32076 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:45.668884 32036 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.707739 32076 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:45.709444 31968 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.709636 32076 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:45.710458 32011 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.821s	user 0.011s	sys 0.000s
W20260709 13:11:45.699710 32040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.318s	user 0.015s	sys 0.000s
W20260709 13:11:45.716280 32040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.337s	user 0.015s	sys 0.000s
I20260709 13:11:45.689874 32102 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.694906 32089 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.733596 32054 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:45.734323 32102 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.725904 32066 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:45.780578 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.731227 32022 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:45.854697 32022 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:45.784225 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 146387us
I20260709 13:11:45.785358 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1789) took 2427 ms. Trace:
I20260709 13:11:45.790287 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.857075 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:45.857668 29077 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:45.859437 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.794126 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:45.816155 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.860543 29081 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 4.
I20260709 13:11:45.861316 32108 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:45.861930 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.828202 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.862659 31957 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:45.863544 31957 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.864281 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.864742 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.865787 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.866559 31957 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:45.866825 29712 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:45.827310 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.829527 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.834004 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.835801 31629 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.739923 32089 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.874953 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.835992 31960 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.705760 31955 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.876624 31960 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.843875 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.841045 31885 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:45.851456 29001 raft_consensus.cc:2368] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8.
I20260709 13:11:45.852165 32036 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.827262 29708 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:11:45.868566 29081 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:45.855971 29003 rpcz_store.cc:276] 0709 13:11:43.357578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.357699 (+   121us) service_pool.cc:224] Handling call
0709 13:11:43.358020 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:43.375620 (+ 17600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.375654 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:43.405674 (+ 30020us) log.cc:844] Serialized 26 byte log entry
0709 13:11:43.667993 (+262319us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:43.668036 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.668127 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:44.758335 (+1090208us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b48000801d8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411a76aaa1 00007f411a761b37 00007f411a7619ca 00007f411a85926f 00007f411a858f37 00007f411a86d55c 00007f411a86d508 00007f411a86d488 00007f411a86d443 00007f411a86c6ec 00007f4119796a96
0709 13:11:45.709901 (+951566us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:45.784559 (+ 74658us) raft_consensus.cc:1685] finished
0709 13:11:45.784641 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.785056 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55636,"raft.queue_time_us":17718,"raft.run_cpu_time_us":3506,"raft.run_wall_time_us":1380011,"spinlock_wait_cycles":43699840,"thread_start_us":231127,"threads_started":1,"wal-append.queue_time_us":433783}
I20260709 13:11:45.699661 31844 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:45.872085 29051 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:45.857625 32118 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:11:45.855567 32022 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.866580 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.843981 31823 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:45.905645 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.901289 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.899408 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:45.926653 29082 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:45.912403 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 61615us
I20260709 13:11:45.897239 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.899904 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.880522 29714 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:45.918840 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1676) took 1051 ms. Trace:
W20260709 13:11:45.937060 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:45.937552 31876 consensus_meta.cc:287] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.553s	user 0.004s	sys 0.016s
I20260709 13:11:45.914541 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.938623 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1388) took 2086 ms. Trace:
I20260709 13:11:45.954458 29078 rpcz_store.cc:276] 0709 13:11:44.866930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.867088 (+   158us) service_pool.cc:224] Handling call
0709 13:11:45.918295 (+1051207us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400404554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:45.918524 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293400960}
I20260709 13:11:45.958561 29711 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:45.917366 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:45.963722 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:45.883826 31981 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.003s
I20260709 13:11:45.965090 31981 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:11:45.965728 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.966873 29076 raft_consensus.cc:2410] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:11:45.941005 32076 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.962241 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.956278 31876 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.970866 29001 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:45.971598 29715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 641) took 2096 ms. Trace:
I20260709 13:11:45.972513 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:45.973264 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1382) took 2331 ms. Trace:
I20260709 13:11:45.973852 29151 rpcz_store.cc:276] 0709 13:11:43.641291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.690479 (+ 49188us) service_pool.cc:224] Handling call
0709 13:11:43.859231 (+168752us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:43.859951 (+   720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.961935 (+2101984us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54002a2bd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:45.972989 (+ 11054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303973120}
I20260709 13:11:45.977380 32076 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:45.959134 29150 rpcz_store.cc:276] 0709 13:11:43.852432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.860559 (+  8127us) service_pool.cc:224] Handling call
0709 13:11:43.860927 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:44.633267 (+772340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.937977 (+1304710us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b54000951d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:45.938384 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1753316736}
W20260709 13:11:45.978994 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.979126 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:45.971959 29715 rpcz_store.cc:276] 0709 13:11:43.875295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.875420 (+   125us) service_pool.cc:224] Handling call
0709 13:11:43.889734 (+ 14314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.748226 (+858492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.971116 (+1222890us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b5400612bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:45.971400 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922961152}
I20260709 13:11:45.919785 32118 consensus_queue.cc:1243] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:11:45.983418 32014 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.006s
I20260709 13:11:45.881240 31885 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:45.984052 32014 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:45.985986 29085 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:45.898239 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:45.986981 31885 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:45.912019 32029 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:45.996790 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:46.001570 29003 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:46.002183 31980 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:46.002681 29007 raft_consensus.cc:2368] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7.
I20260709 13:11:45.996333 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:46.003645 29711 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:46.003719 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:46.004951 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:45.999451 29077 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:11:46.007050 29150 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:45.879170 31955 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:46.008509 29077 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:46.006716 29084 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:11:45.879870 31960 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:46.013548 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 154158us
I20260709 13:11:46.015203 29048 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:46.018302 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:46.020521 29051 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:46.002552 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:46.022132 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:46.029528 31908 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:46.030764 29122 leader_election.cc:400] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7.
I20260709 13:11:45.997027 31876 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:45.926440 32022 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.031122 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:46.005069 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:45.998775 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:46.036294 29122 leader_election.cc:403] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:46.027733 29077 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:46.036734 31908 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:46.043423 29010 raft_consensus.cc:2410] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:11:46.043979 32020 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:11:46.044821 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:46.046245 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.053839 32020 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7.
W20260709 13:11:46.054263 31773 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.565s	user 0.005s	sys 0.014s
I20260709 13:11:46.055472 29079 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:11:46.055938 29079 pending_rounds.cc:85] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 3
I20260709 13:11:46.056545 29079 pending_rounds.cc:107] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:11:46.056890 29079 raft_consensus.cc:2889] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:46.056931 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:46.058125 29150 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:46.061394 32102 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:11:46.064179 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.044056 31908 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:46.076867 31908 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:46.081508 29001 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
W20260709 13:11:46.083253 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 68208.6us
I20260709 13:11:46.075829 31824 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.987097 32014 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.087564 31824 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.091109 31856 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:46.091785 32118 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:46.094682 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:46.096287 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 76031.7us
I20260709 13:11:46.097589 29051 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:46.099272 32025 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:46.099378 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:46.100010 32025 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:46.100351 29003 raft_consensus.cc:2368] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7.
I20260709 13:11:46.101239 32025 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.104132 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:46.105158 29156 raft_consensus.cc:2410] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:11:46.106983 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1394) took 1735 ms. Trace:
I20260709 13:11:46.107482 29158 rpcz_store.cc:276] 0709 13:11:44.371446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.526368 (+154922us) service_pool.cc:224] Handling call
0709 13:11:44.601671 (+ 75303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.601848 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.601871 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.601907 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:44.601934 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.601985 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:46.106551 (+1504566us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b5400139f50. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:46.106770 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139688832}
I20260709 13:11:46.112293 31025 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.113897 29010 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:46.063441 32098 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:46.119987 29007 raft_consensus.cc:2410] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:46.085101 31773 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:46.139210 31767 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:46.141880 31767 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:46.152635 29009 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:11:46.155208 29712 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:46.155921 31025 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.097608 32064 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.153234 29009 pending_rounds.cc:85] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 3
I20260709 13:11:46.163447 29009 pending_rounds.cc:107] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:11:46.163705 29009 raft_consensus.cc:2889] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:46.166822 31823 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:46.168813 31767 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.169401 31767 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.183660 29082 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:46.183602 29078 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:46.186832 31999 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:46.189426 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:46.191623 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:46.191779 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:46.193007 32112 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:46.192940 32090 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:46.194151 29122 leader_election.cc:400] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7.
I20260709 13:11:46.194474 29122 leader_election.cc:403] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:11:46.196060 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 68907.6us
W20260709 13:11:46.197365 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.198393 29077 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:46.165501 32118 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:46.219754 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:46.221769 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:46.225060 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 86882.3us
I20260709 13:11:46.226527 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:46.223784 32118 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:46.232913 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 449495us
I20260709 13:11:46.240875 29158 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:46.244609 32118 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:46.246173 31907 raft_consensus.cc:3055] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:46.246443 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:46.246663 31907 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:46.252395 31025 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:46.253252 29004 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:46.254806 29004 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:46.255005 31025 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:46.255918 31629 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:46.256310 31629 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:46.256958 32045 consensus_meta.cc:287] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.599s	user 0.013s	sys 0.009s
I20260709 13:11:46.257143 31629 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.257416 32045 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.258570 31629 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:46.259678 29003 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:46.268116 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:46.268885 29155 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:46.269109 32118 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:46.247701 31907 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:46.271786 32016 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:46.277174 32118 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:46.251726 31965 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:46.282647 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44683.6us
I20260709 13:11:46.233583 32020 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:11:46.277179 31907 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:46.300477 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 104696us
I20260709 13:11:46.294498 31824 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.294958 32092 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.301319 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:46.301537 32092 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:46.302628 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:46.305470 32092 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.307868 31824 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:46.308876 31824 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:46.309262 31824 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:46.309695 31824 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:46.310309 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:46.321653 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.318629 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:46.317107 29156 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:46.357363 29010 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:46.357920 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 47873.5us
W20260709 13:11:46.359056 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.359483 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:46.352175 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:46.360536 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:46.354609 32045 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:46.501942 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.503715 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:46.504554 31890 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:46.504773 31968 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:46.505254 31890 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.505522 31968 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:46.506160 31968 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:46.507479 31890 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:46.515811 32024 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:46.516695 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:46.517328 29152 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:46.520383 32005 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:46.521034 32005 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.523267 32005 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:46.515976 32023 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:46.537245 29155 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:46.549413 29150 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:46.481165 32043 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.006s
W20260709 13:11:46.601073 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 222931us
W20260709 13:11:46.610096 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51864us
I20260709 13:11:46.634326 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1316) took 1103 ms. Trace:
W20260709 13:11:46.637881 32125 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 13:11:46.639192 29005 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:46.640297 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.535796 32023 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:46.634900 29081 rpcz_store.cc:276] 0709 13:11:45.531320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.861505 (+330185us) service_pool.cc:224] Handling call
0709 13:11:46.625294 (+763789us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b54000c94d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:46.634267 (+  8973us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13328128}
I20260709 13:11:46.648141 29002 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
W20260709 13:11:46.648264 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 344235us
I20260709 13:11:46.652431 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:46.572539 32098 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.675127 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:46.678143 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 354138us
W20260709 13:11:46.675320 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 102081us
I20260709 13:11:46.647881 31981 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.648099 32025 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:46.696815 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:46.697445 32126 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:11:46.685426 31907 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.702205 31907 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:46.705004 31907 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:46.705909 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44814.5us
I20260709 13:11:46.706796 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:46.708204 31957 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:46.709208 31823 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:46.709622 31823 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:46.709992 31823 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:46.708006 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:46.715714 31981 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.693233 32025 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:46.686879 31824 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:46.738842 29152 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
W20260709 13:11:46.740020 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 47892.4us
I20260709 13:11:46.744434 29158 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:46.746065 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:46.723834 32118 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.727926 32091 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:46.759819 28976 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:46.769196 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:46.770057 32130 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:11:46.770208 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:46.770738 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:46.771214 29709 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:46.771589 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:46.771993 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:46.772218 29085 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:46.773478 29085 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:46.774762 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:46.783905 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 47920.9us
I20260709 13:11:46.775836 29082 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:46.776676 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:46.777405 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:46.778527 29714 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:46.738835 31773 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:11:46.779563 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52208.3us
I20260709 13:11:46.780102 32043 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:46.737207 32025 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:46.778590 32129 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:11:46.791191 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1302) took 1424 ms. Trace:
I20260709 13:11:46.791654 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:46.798959 31910 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.799113 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:46.797683 29084 rpcz_store.cc:276] 0709 13:11:45.367139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.426560 (+ 59421us) service_pool.cc:224] Handling call
0709 13:11:46.781621 (+1355061us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b540034ec54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:46.791134 (+  9513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185371392}
I20260709 13:11:46.800102 29081 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:46.801875 29712 raft_consensus.cc:2368] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7.
I20260709 13:11:46.790344 29710 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:46.808419 31824 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.808198 31968 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:46.813895 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 105952us
I20260709 13:11:46.816193 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:46.817216 29085 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:46.787765 31773 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:46.790098 32043 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:46.779683 32023 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:46.830785 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:46.822793 32020 consensus_meta.cc:287] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.527s	user 0.004s	sys 0.005s
I20260709 13:11:46.787303 32118 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:46.861027 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.827105 31773 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:46.827411 32043 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:46.867653 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 171121us
I20260709 13:11:46.809499 31860 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:11:46.877477 29009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.077s	sys 0.169s
W20260709 13:11:46.883606 29009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.077s	sys 0.169s
I20260709 13:11:46.882615 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:46.828569 31974 consensus_meta.cc:287] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.100s	user 0.017s	sys 0.002s
W20260709 13:11:46.832198 32132 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:11:46.824365 31767 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.891367 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 409) took 1036 ms. Trace:
I20260709 13:11:46.890740 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:46.903262 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:46.906890 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.882071 32023 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:46.918402 29078 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:46.920930 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1816) took 1143 ms. Trace:
I20260709 13:11:46.921320 29004 rpcz_store.cc:276] 0709 13:11:45.777197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.255516 (+478319us) service_pool.cc:224] Handling call
0709 13:11:46.255883 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:46.277525 (+ 21642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:46.277631 (+   106us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:46.278153 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:11:46.661467 (+383314us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:46.661536 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.661626 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:46.920728 (+259102us) raft_consensus.cc:1685] finished
0709 13:11:46.920746 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:46.920874 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":391,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":125,"thread_start_us":382767,"threads_started":1,"wal-append.queue_time_us":604031}
I20260709 13:11:46.895138 32020 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7.
I20260709 13:11:46.825181 32024 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.086s
W20260709 13:11:46.937714 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 116207us
I20260709 13:11:46.939489 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:46.945971 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:46.887809 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:46.927906 31824 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.949169 29152 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:46.950553 29155 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:46.884819 31910 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:46.953359 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 121320us
I20260709 13:11:46.955765 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:46.965497 29084 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:46.951318 31968 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:46.948137 32091 consensus_queue.cc:1059] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:11:46.967320 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:46.969089 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:46.972672 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:46.973757 29002 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:46.975783 32140 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:11:46.972760 32098 consensus_queue.cc:1059] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
W20260709 13:11:46.982086 32133 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.341s	user 0.004s	sys 0.000s
I20260709 13:11:46.986729 29710 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:46.960124 31974 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:46.990515 32118 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:46.991226 32118 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:46.991654 32118 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:46.953486 31981 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.009708 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:47.011667 29082 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:47.016523 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:47.018213 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.026604 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:46.997999 32118 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:47.035686 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:47.039822 29004 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:47.041218 29156 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:46.961028 31824 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:47.048350 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:47.049738 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:47.050451 29714 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:47.061602 29151 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:47.073290 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:46.999521 31960 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:11:47.085390 31455 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:47.085961 31455 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:47.087225 29716 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:11:47.111891 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.118183 29153 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:47.183998 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:47.184672 29155 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:47.185925 29152 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:47.101316 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2128) took 1590 ms. Trace:
I20260709 13:11:47.090601 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:47.102995 32008 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:47.061767 31981 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 13:11:47.200245 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 441) took 1085 ms. Trace:
I20260709 13:11:47.187168 29006 rpcz_store.cc:276] 0709 13:11:45.441327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.443297 (+  1970us) service_pool.cc:224] Handling call
0709 13:11:46.960598 (+1517301us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400280354. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:47.031553 (+ 70955us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401104768}
I20260709 13:11:47.210044 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:47.103546 32134 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:11:47.206061 29079 rpcz_store.cc:276] 0709 13:11:46.068283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.089949 (+ 21666us) service_pool.cc:224] Handling call
0709 13:11:46.634267 (+544318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:46.642080 (+  7813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:46.642110 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:46.642169 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:46.642200 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.642253 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:47.153887 (+511634us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54000c94d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:47.154030 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250860288}
I20260709 13:11:47.216941 29156 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
W20260709 13:11:47.114142 32131 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.473s	user 0.000s	sys 0.004s
W20260709 13:11:47.111027 32136 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.469s	user 0.000s	sys 0.002s
I20260709 13:11:47.081179 32142 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.158092 31824 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:47.087982 31455 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:47.098965 31960 consensus_queue.cc:1243] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:47.225955 29004 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:47.219746 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:47.097714 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:47.099212 31860 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:47.128122 31968 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:47.156105 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:47.100471 31976 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:47.222910 29714 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:47.242859 31968 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:47.238184 29708 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:47.125102 32025 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:47.208771 31974 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:11:47.254077 29159 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:47.256831 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 166374us
I20260709 13:11:47.219126 31773 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:47.267083 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.241318 31860 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.309688 29150 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:47.253914 32048 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:47.256345 31999 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:47.259558 29125 connection.cc:256] RPC call timeout handler was delayed by -1.1097624450000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
W20260709 13:11:47.315543 29125 connection.cc:256] RPC call timeout handler was delayed by -1.0053624449999461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
I20260709 13:11:47.236173 32142 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:47.310783 31823 consensus_meta.cc:287] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.006s
I20260709 13:11:47.264739 31968 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:47.238667 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.327771 31823 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:47.332935 31823 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:47.333896 32118 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.317696 31999 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.336288 31999 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:47.336565 32118 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.336872 32025 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:47.336925 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:47.337564 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:47.331004 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:47.338851 29008 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:47.339704 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:47.340526 31890 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:47.340911 32142 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:47.341348 29085 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:47.335837 31860 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:47.312857 32048 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:47.263707 31976 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:47.345388 29159 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
W20260709 13:11:47.349650 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 380598us
I20260709 13:11:47.324476 31910 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.359560 29156 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:47.361363 29125 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:47.363684 31529 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:47.364789 31890 raft_consensus.cc:2764] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:11:47.365167 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:47.366562 32024 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:47.348982 32048 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:47.368137 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:47.370061 31629 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:47.349258 32092 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:47.375406 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.383018 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:47.385064 29156 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:47.389205 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:47.394734 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:47.395838 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.399103 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.400638 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.401566 29156 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:47.404703 29082 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:47.405651 29048 connection.cc:256] RPC call timeout handler was delayed by -1.429130389000079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
W20260709 13:11:47.406208 29048 connection.cc:256] RPC call timeout handler was delayed by -1.2239303890000883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
I20260709 13:11:47.406663 29008 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:11:47.406598 29048 connection.cc:256] RPC call timeout handler was delayed by -1.1726303890000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
I20260709 13:11:47.376958 32092 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:47.412266 32064 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:47.399379 32142 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:11:47.417476 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 207483us
I20260709 13:11:47.419315 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.419514 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.421728 29159 raft_consensus.cc:2410] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:47.425086 31773 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:47.425683 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:47.425751 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:47.426741 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.434873 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.437014 29008 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:47.438755 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:47.417987 32142 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:47.447121 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.451673 31529 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:47.454731 31976 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:47.455482 32043 consensus_meta.cc:287] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.002s
I20260709 13:11:47.439570 31999 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:47.462693 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:47.473060 29159 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:47.478628 29003 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
W20260709 13:11:47.478907 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 59829.4us
I20260709 13:11:47.438345 31968 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.483377 31975 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:47.483559 29005 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:47.483870 31975 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:47.484369 32092 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.484831 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.485482 29158 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:47.486017 31999 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.486671 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:47.487370 29710 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:46.895081 29009 rpcz_store.cc:276] 0709 13:11:45.855568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.260306 (+404738us) service_pool.cc:224] Handling call
0709 13:11:46.260678 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:46.260958 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:46.261251 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 13:11:46.891021 (+629770us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:46.891070 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:46.891121 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.891183 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:46.891299 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":506,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":10842,"thread_start_us":622445,"threads_started":1}
W20260709 13:11:47.490634 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 65015.3us
I20260709 13:11:47.464522 32142 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:47.493788 29682 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:47.495397 29159 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:11:47.502694 29154 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:47.492318 32005 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:47.496940 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:47.483389 31968 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.512585 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.509595 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:47.489990 32043 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.520417 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:47.506244 31907 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:47.522377 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.522245 31529 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:47.523145 29157 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:47.523083 32147 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:11:47.524179 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:47.525381 31773 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:47.528743 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:47.506692 31960 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:11:47.528405 31860 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:47.535869 32020 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:47.540633 32023 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:47.540797 32020 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:47.541190 32023 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:47.541865 32020 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.542155 32023 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.542906 32020 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:47.542971 31921 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:47.540948 31860 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:47.545994 31860 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.536104 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:47.547581 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52344us
I20260709 13:11:47.538857 31529 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:47.548242 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:47.538384 29125 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:47.551132 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 54271us
I20260709 13:11:47.551877 29001 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:47.551091 29155 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:11:47.543779 32023 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:47.553004 29157 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:47.539736 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:47.530433 32091 raft_consensus.cc:3055] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:47.552165 31773 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.569205 31773 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.570155 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:47.571646 31529 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:47.563933 31910 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:47.572752 31910 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.574474 31999 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:47.574664 31910 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.574976 32048 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.575489 32151 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:47.576093 29006 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:47.576385 31968 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:47.580137 32064 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:47.583148 31974 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:47.583876 31974 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:47.584836 31974 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.534403 32167 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 13:11:47.569095 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 127671us
W20260709 13:11:47.593825 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 73500.4us
I20260709 13:11:47.565850 32091 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:47.587894 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.847s (SENT) [suppressed 13 similar messages]
I20260709 13:11:47.596863 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:47.599195 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.576083 32151 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:47.612771 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.602200 31455 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:11:47.622982 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.628892 29083 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:47.625862 29154 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:47.603493 32147 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:11:47.632477 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.522388 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:47.639256 32048 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.640787 32024 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:47.644058 32043 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:47.644631 32151 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.644652 32159 consensus_queue.cc:1059] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:11:47.645957 32159 raft_consensus.cc:3055] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:47.646391 32159 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:47.663949 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1285) took 3388 ms. Trace:
I20260709 13:11:47.637826 32150 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:47.622193 32167 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:11:47.633872 32168 consensus_queue.cc:1059] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 13:11:47.698130 29083 rpcz_store.cc:276] 0709 13:11:44.275606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.278144 (+  2538us) service_pool.cc:224] Handling call
0709 13:11:44.278360 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.629282 (+3350922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.663603 (+ 34321us) spinlock_profiling.cc:243] Waited 3.33 s on lock 0x7b5400023d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:47.663853 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7655838592}
I20260709 13:11:47.646433 31975 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:47.822480 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.698107 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 101296us
W20260709 13:11:47.816876 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 119497us
I20260709 13:11:47.824337 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:47.826323 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:47.839308 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:47.816213 32005 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:47.844012 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1441) took 1503 ms. Trace:
I20260709 13:11:47.877017 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:47.878190 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 290286us
I20260709 13:11:47.884690 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:47.885779 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:47.887948 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 289006us
I20260709 13:11:47.890163 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:47.891151 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:47.871610 29153 rpcz_store.cc:276] 0709 13:11:46.325264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.325421 (+   157us) service_pool.cc:224] Handling call
0709 13:11:47.655722 (+1330301us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54000942d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:47.828398 (+172676us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1027676416}
I20260709 13:11:47.894132 29008 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:47.877934 32005 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:47.920002 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.919989 31975 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:47.932546 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:47.846257 31823 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:47.937000 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:47.892817 32167 consensus_queue.cc:1243] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:11:47.892889 31773 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:47.951817 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.896824 32118 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:47.949154 29682 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:47.962790 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:47.963591 29709 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:47.963968 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:47.920986 31960 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:47.954324 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:47.967883 29083 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:47.966226 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:47.970611 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:47.971068 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:47.970157 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:47.972972 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:47.965081 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:47.977269 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:47.964355 32150 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:47.953835 31921 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.969559 31960 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:47.974665 32171 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:11:47.963044 32118 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:48.021221 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.030829 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 393327us
I20260709 13:11:47.971747 32172 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:47.975229 32016 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:47.980096 31999 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:48.078621 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 188485us
I20260709 13:11:47.964169 31823 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:47.967154 31860 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
W20260709 13:11:48.080303 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 242055us
I20260709 13:11:47.966898 31773 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.002585 32150 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:47.990550 31455 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:47.939563 32159 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.063760 31629 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:11:48.100644 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 127743us
I20260709 13:11:48.075129 32118 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:48.103003 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 210369us
I20260709 13:11:48.071316 32172 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:48.070667 31960 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:48.083801 31999 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.115521 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 447) took 1193 ms. Trace:
I20260709 13:11:48.116638 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:48.101895 32150 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:48.117898 29085 rpcz_store.cc:276] 0709 13:11:46.922210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.931432 (+  9222us) service_pool.cc:224] Handling call
0709 13:11:46.937376 (+  5944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.341746 (+404370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.115146 (+773400us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540025a154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:48.115435 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928498944}
W20260709 13:11:48.124632 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 158464us
I20260709 13:11:48.076092 31921 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.118914 32172 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:48.115546 32142 consensus_meta.cc:287] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.599s	user 0.011s	sys 0.003s
W20260709 13:11:48.117578 32139 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.075s	user 0.002s	sys 0.008s
I20260709 13:11:48.132454 29125 leader_election.cc:400] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7.
I20260709 13:11:48.105545 31629 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:48.120543 31999 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:48.132326 32020 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.589s	user 0.018s	sys 0.000s
I20260709 13:11:48.123287 32183 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:48.134338 32064 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:48.117266 31529 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:48.115231 31455 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.181077 32159 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:48.231422 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 99918.6us
I20260709 13:11:48.232285 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:48.257840 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1828) took 1058 ms. Trace:
I20260709 13:11:48.209923 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:48.258250 29001 rpcz_store.cc:276] 0709 13:11:47.172718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.172833 (+   115us) service_pool.cc:224] Handling call
0709 13:11:47.173124 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.552265 (+379141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.230480 (+678215us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54003ff554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:48.230674 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870593280}
I20260709 13:11:48.137240 32142 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.261323 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:48.274003 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 171539us
W20260709 13:11:48.285234 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 53024.5us
I20260709 13:11:48.286289 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:48.289638 32177 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:11:48.290325 32142 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:48.291306 32142 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:48.291479 32173 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.350s	user 0.001s	sys 0.000s
I20260709 13:11:48.323747 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:48.257445 32179 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.308s
I20260709 13:11:48.292439 31629 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.297423 31455 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:48.273741 31921 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:48.271906 32023 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.708s	user 0.005s	sys 0.005s
I20260709 13:11:48.198869 32183 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.298174 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:48.298369 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:48.299423 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:48.339105 29079 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
W20260709 13:11:48.311898 32181 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:11:48.340363 31455 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:48.341106 31455 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:48.312824 32175 log.cc:927] Time spent T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:11:48.342247 31455 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.320294 32091 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.206429 32192 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:48.349076 32190 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:48.349959 32189 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:48.350909 32182 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:48.352406 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.354571 31773 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:48.356225 32174 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
I20260709 13:11:48.310832 28905 catalog_manager.cc:5697] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 2 to 3, leader changed from <none> to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:48.274833 31529 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.285094 32191 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
W20260709 13:11:48.366648 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 278544us
I20260709 13:11:48.332610 32163 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.390022 31773 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:48.372344 29078 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:11:48.357764 32176 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 13:11:48.375315 32190 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:48.432879 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 146679us
I20260709 13:11:48.395241 32091 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:48.393676 32020 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:48.442869 29051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.001s	sys 0.014s
I20260709 13:11:48.448442 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 370) took 1109 ms. Trace:
W20260709 13:11:48.450893 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 255879us
I20260709 13:11:48.449069 29159 rpcz_store.cc:276] 0709 13:11:47.338706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.499861 (+161155us) service_pool.cc:224] Handling call
0709 13:11:47.500171 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:47.525039 (+ 24868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:47.525070 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:47.525514 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:11:47.624008 (+ 98494us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:47.624065 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.624130 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:48.448141 (+824011us) raft_consensus.cc:1685] finished
0709 13:11:48.448173 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:48.448362 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31071,"raft.queue_time_us":5415,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":22638,"thread_start_us":98117,"threads_started":1,"wal-append.queue_time_us":112872}
I20260709 13:11:48.455236 28975 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:48.390404 32182 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:48.455482 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 131819us
W20260709 13:11:48.454587 29051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.001s	sys 0.014s
I20260709 13:11:48.456507 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.459149 29078 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:48.462445 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 542302us
W20260709 13:11:48.467658 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.475455 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 202883us
I20260709 13:11:48.446695 32064 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:48.445415 32024 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.485046 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.479200 29002 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:48.489169 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.490710 28975 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:48.493639 29010 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:48.495565 29077 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:11:48.499588 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:48.505770 29009 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:48.477946 32186 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:48.477416 32184 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.497936 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:48.516763 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 76220.4us
W20260709 13:11:48.520655 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 88300.8us
I20260709 13:11:48.436414 32163 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.456425 32020 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.477631 32197 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:48.541455 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:48.542237 29709 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:48.542364 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:48.542431 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:48.543987 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.544798 29085 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:48.538204 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 88761.7us
I20260709 13:11:48.538450 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:48.545377 29680 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:48.519817 32200 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:11:48.553653 29716 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:48.554076 29009 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:48.553364 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.556432 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:48.556488 29708 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 4.
I20260709 13:11:48.557226 29711 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:48.520219 32167 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:48.555332 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:48.560812 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.561626 29083 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:48.563935 29715 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:48.517858 32191 consensus_queue.cc:1059] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:11:48.492868 32024 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.570443 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:48.571648 32191 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:48.571868 32024 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:48.571759 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.572902 29079 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:48.573136 32191 raft_consensus.cc:3055] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:48.573603 32191 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:48.573797 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.574442 32191 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.575067 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.575680 32191 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.576274 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:48.579334 32216 consensus_queue.cc:1059] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:48.580276 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:48.580550 32196 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:11:48.583050 29080 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:11:48.584373 32195 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:11:48.584502 29080 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:11:48.587481 29085 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
W20260709 13:11:48.588438 28976 connection.cc:256] RPC call timeout handler was delayed by -2.0847869950000586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2388s for a response
W20260709 13:11:48.589211 28976 connection.cc:256] RPC call timeout handler was delayed by -1.9284900310000239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 13:11:48.564385 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:11:48.589684 28976 connection.cc:256] RPC call timeout handler was delayed by -1.8411900310001101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 13:11:48.590200 28976 connection.cc:256] RPC call timeout handler was delayed by -1.8213900310000781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 13:11:48.591281 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 115040us
W20260709 13:11:48.590711 28976 connection.cc:256] RPC call timeout handler was delayed by -1.7754900310001176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 13:11:48.529309 31860 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:11:48.562526 32167 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.602177 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:48.599198 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6458900310000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
W20260709 13:11:48.603686 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6017900310000641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 13:11:48.604604 28976 connection.cc:256] RPC call timeout handler was delayed by -1.4620918630000688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
W20260709 13:11:48.605108 28976 connection.cc:256] RPC call timeout handler was delayed by -1.3929900310000676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21110000000000004s for a response
W20260709 13:11:48.605571 28976 connection.cc:256] RPC call timeout handler was delayed by -1.3686900310000283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
W20260709 13:11:48.606011 28976 connection.cc:256] RPC call timeout handler was delayed by -1.3119900310000503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
W20260709 13:11:48.606465 28976 connection.cc:256] RPC call timeout handler was delayed by -1.2900869950000242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3271s for a response
W20260709 13:11:48.606895 28976 connection.cc:256] RPC call timeout handler was delayed by -1.2730918630001042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
W20260709 13:11:48.607326 28976 connection.cc:256] RPC call timeout handler was delayed by -1.1706900310000492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2358s for a response
W20260709 13:11:48.618345 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.591184 32023 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.529698 32184 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.534101 32186 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.653548 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:48.608338 32167 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:48.669338 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:48.670635 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 145862us
I20260709 13:11:48.635128 32183 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:48.676198 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 110769us
I20260709 13:11:48.676919 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:48.678483 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:48.680301 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:48.653147 31529 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:48.679165 29005 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:48.685478 29010 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:11:48.685966 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.692234 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:48.652863 32023 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.564608 32190 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:48.682916 32219 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:48.685657 32202 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:48.722748 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:48.723752 29005 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:48.648375 32064 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:48.690131 32200 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:11:48.688815 32183 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:11:48.769969 29010 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:11:48.714769 32224 raft_consensus.cc:3055] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:11:48.770767 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 214888us
I20260709 13:11:48.837913 32224 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:48.848605 32227 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 13:11:48.778530 32220 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:11:48.852872 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:48.854640 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:48.931437 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:48.909385 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.963195 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 126174us
I20260709 13:11:48.936794 29077 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:48.764669 32215 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:49.001093 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:49.030862 29010 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:48.956979 31975 consensus_meta.cc:287] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.013s	user 0.012s	sys 0.000s
I20260709 13:11:49.034463 29002 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:49.000145 29081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.020s	sys 0.030s
W20260709 13:11:49.044242 29081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.020s	sys 0.030s
I20260709 13:11:49.045212 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1702) took 1099 ms. Trace:
I20260709 13:11:48.755795 32182 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:48.770633 32020 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:48.848639 32223 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:49.049383 29008 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:48.962203 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:48.764859 32219 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:49.293334 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:49.230631 29051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.007s	sys 0.023s
I20260709 13:11:48.892700 32210 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:48.963267 32206 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:49.192251 32005 consensus_meta.cc:287] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.240s	user 0.006s	sys 0.008s
I20260709 13:11:49.343327 32005 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.295398 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:49.230942 29080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.007s	sys 0.021s
W20260709 13:11:49.066465 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 367539us
I20260709 13:11:49.296530 29010 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:49.181723 29709 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
W20260709 13:11:49.226120 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 234710us
W20260709 13:11:49.123631 29078 consensus_meta.cc:287] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.610s	user 0.008s	sys 0.000s
I20260709 13:11:49.061522 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:49.299260 29711 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.011s
I20260709 13:11:49.045665 29081 rpcz_store.cc:276] 0709 13:11:47.945507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.945644 (+   137us) service_pool.cc:224] Handling call
0709 13:11:48.325935 (+380291us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:48.326198 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:48.326505 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:48.326872 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.044788 (+717916us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:49.044856 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:49.044913 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.044995 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.045145 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":198,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":46,"spinlock_wait_cycles":2935552,"thread_start_us":717419,"threads_started":1}
I20260709 13:11:49.099256 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:49.118049 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:49.287672 32191 consensus_meta.cc:287] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.000s
W20260709 13:11:49.289376 32221 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.396s	user 0.001s	sys 0.000s
I20260709 13:11:49.354537 32213 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:49.357744 32172 consensus_meta.cc:287] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.229s	user 0.007s	sys 0.005s
I20260709 13:11:49.357532 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:49.334282 32224 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:49.359217 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.346474 32223 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:49.294039 29051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.008s	sys 0.025s
W20260709 13:11:49.586354 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 1.1094e+06us
W20260709 13:11:49.190893 31999 consensus_meta.cc:287] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.934s	user 0.010s	sys 0.000s
W20260709 13:11:49.361804 29080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.011s	sys 0.021s
W20260709 13:11:49.413595 29082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.003s	sys 0.022s
I20260709 13:11:49.517335 29711 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
W20260709 13:11:49.587838 29082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.006s	sys 0.032s
I20260709 13:11:49.588186 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 426) took 1047 ms. Trace:
I20260709 13:11:49.362355 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:49.589599 29010 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:49.191736 32150 consensus_meta.cc:287] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.009s	user 0.005s	sys 0.004s
I20260709 13:11:49.518491 29007 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:49.519881 29078 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:49.523620 32230 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.521016 29005 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:49.432163 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.634414 29008 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:11:49.492198 29050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.842s	user 0.000s	sys 0.012s
W20260709 13:11:49.635285 29050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.000s	sys 0.013s
W20260709 13:11:49.635601 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 1.03887e+06us
I20260709 13:11:49.034168 32202 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:49.379748 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:49.140686 32170 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:49.296434 32023 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.189739 32184 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:49.128352 32222 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:49.404490 32118 consensus_meta.cc:287] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.282s	user 0.010s	sys 0.003s
I20260709 13:11:49.671777 32222 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.188923 32163 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.588707 29711 rpcz_store.cc:276] 0709 13:11:48.540565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.540774 (+   209us) service_pool.cc:224] Handling call
0709 13:11:49.588101 (+1047327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.334797 32168 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:49.600610 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1324) took 1146 ms. Trace:
I20260709 13:11:49.675455 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.675748 29078 rpcz_store.cc:276] 0709 13:11:48.454134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.456183 (+  2049us) service_pool.cc:224] Handling call
0709 13:11:49.600542 (+1144359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.129364 32236 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:49.603617 32241 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.600272 29002 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
W20260709 13:11:49.650625 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 270913us
I20260709 13:11:49.679162 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.414490 32207 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.171s
W20260709 13:11:49.593009 31960 consensus_meta.cc:287] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.467s	user 0.007s	sys 0.007s
I20260709 13:11:49.621860 32250 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:49.622292 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 264847us
I20260709 13:11:49.592911 32235 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:49.622550 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.690933 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.691250 29007 raft_consensus.cc:2410] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:11:49.623292 29010 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:11:49.692394 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.693130 29010 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:49.693670 29002 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:49.693832 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.694585 29010 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:49.625144 32091 consensus_meta.cc:287] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.176s	user 0.008s	sys 0.003s
W20260709 13:11:49.627910 32239 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 13:11:49.628034 32167 consensus_meta.cc:287] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.952s	user 0.007s	sys 0.000s
I20260709 13:11:49.607031 31999 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.593102 32092 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.632148 32223 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.638116 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.607252 32208 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.645521 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:49.648371 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.671851 32118 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.039161 32215 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.402333 32206 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.641338 32005 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:49.592651 32172 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.697315 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.677912 32023 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:49.749145 32253 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:49.748900 32167 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.607465 32150 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.691707 32207 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.593617 32213 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.812862 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:49.816207 29007 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:49.827822 29084 raft_consensus.cc:2368] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6.
I20260709 13:11:49.674631 32222 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:49.647502 32230 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.690519 31960 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.859748 31999 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.687294 32168 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.863209 32168 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.811703 32208 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.869431 32206 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.869830 29713 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:49.752892 32092 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.871352 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 432) took 1326 ms. Trace:
I20260709 13:11:49.871803 29713 rpcz_store.cc:276] 0709 13:11:48.544653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.552781 (+  8128us) service_pool.cc:224] Handling call
0709 13:11:49.871169 (+1318388us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54004e2354. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:49.871272 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2671363840}
I20260709 13:11:49.873483 32092 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.875993 29714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 465) took 1903 ms. Trace:
I20260709 13:11:49.877655 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2150) took 1028 ms. Trace:
I20260709 13:11:49.878460 29714 rpcz_store.cc:276] 0709 13:11:47.972327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.972451 (+   124us) service_pool.cc:224] Handling call
0709 13:11:49.875290 (+1902839us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:11:49.875469 (+   179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.875667 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.875690 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:49.875728 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:49.875761 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.875823 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.875937 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13081344}
I20260709 13:11:49.687485 32236 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.899991 29010 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:49.897374 29001 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:11:49.867719 32208 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.910494 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.688938 32241 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.875993 32210 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:49.921241 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 275794us
I20260709 13:11:49.925575 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:49.892732 32232 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:49.890920 32150 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:49.889943 32023 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:49.909554 32219 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.841092 32253 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.751899 32250 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.909689 32235 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:49.902299 32168 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:49.891258 29006 rpcz_store.cc:276] 0709 13:11:48.848833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.853381 (+  4548us) service_pool.cc:224] Handling call
0709 13:11:48.853745 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:49.225810 (+372065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.225847 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:49.337916 (+112069us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.682097 (+344181us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:49.748524 (+ 66427us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.761724 (+ 13200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.761825 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.877467 (+115642us) raft_consensus.cc:1685] finished
0709 13:11:49.877484 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.877602 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16178,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":2323,"raft.queue_time_us":350296,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":5202,"spinlock_wait_cycles":6192896,"thread_start_us":647745,"threads_started":2,"wal-append.queue_time_us":278854}
I20260709 13:11:49.929651 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:49.945008 29008 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:11:49.842303 32207 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:49.939153 32159 consensus_meta.cc:287] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.681s	user 0.005s	sys 0.009s
I20260709 13:11:49.932237 32232 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:49.949424 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 244003us
W20260709 13:11:49.955184 32247 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:11:49.956319 31960 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:49.956740 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1338) took 1425 ms. Trace:
I20260709 13:11:49.965015 32213 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:49.966346 32118 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:49.928233 32210 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.968247 32241 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.969341 32215 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.972066 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1340) took 1434 ms. Trace:
I20260709 13:11:49.948760 32168 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:49.972472 29079 rpcz_store.cc:276] 0709 13:11:48.538127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.588433 (+ 50306us) service_pool.cc:224] Handling call
0709 13:11:48.601720 (+ 13287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:48.704142 (+102422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:48.704184 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:48.765247 (+ 61063us) log.cc:844] Serialized 52 byte log entry
0709 13:11:49.048578 (+283331us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:49.221980 (+173402us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.222259 (+   279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.222357 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.891997 (+669640us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b1800131800. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:11:49.957111 (+ 65114us) raft_consensus.cc:1685] finished
0709 13:11:49.971815 (+ 14704us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.971986 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20024,"prepare.queue_time_us":174016,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":1544,"raft.queue_time_us":75819,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":17191,"spinlock_wait_cycles":3445120,"thread_start_us":163730,"threads_started":2,"wal-append.queue_time_us":555379}
I20260709 13:11:49.980736 32172 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:49.879061 32202 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:49.959610 32232 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:49.985271 29712 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
W20260709 13:11:49.957149 32243 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:11:49.988545 32167 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.993492 32207 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:49.995352 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:49.995380 29084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1697) took 2058 ms (client timeout 1597 ms). Trace:
W20260709 13:11:50.002688 29084 rpcz_store.cc:269] 0709 13:11:47.936554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.936792 (+   238us) service_pool.cc:224] Handling call
0709 13:11:49.995139 (+2058347us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b540034f154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:49.995310 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4335443328}
I20260709 13:11:50.007655 29008 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
W20260709 13:11:50.012878 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 53513.1us
I20260709 13:11:49.957162 29082 rpcz_store.cc:276] 0709 13:11:48.531295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.586579 (+ 55284us) service_pool.cc:224] Handling call
0709 13:11:48.586878 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.634037 (+ 47159us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:48.675182 (+ 41145us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.616805 (+941623us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:49.617526 (+   721us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.617701 (+   175us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:49.617757 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.617854 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.956553 (+338699us) raft_consensus.cc:1685] finished
0709 13:11:49.956572 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.956687 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69550,"prepare.queue_time_us":24198,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":3874,"raft.queue_time_us":367402,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":7408,"spinlock_wait_cycles":8237440,"thread_start_us":884530,"threads_started":1,"wal-append.queue_time_us":934659}
W20260709 13:11:50.011873 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 186833us
I20260709 13:11:50.016824 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:50.017258 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:50.017843 29711 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:50.017385 29713 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:11:50.019054 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:50.024290 29711 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:50.024695 29713 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
W20260709 13:11:50.036551 32248 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.220s	user 0.000s	sys 0.002s
I20260709 13:11:50.037002 32207 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:50.037958 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:50.013810 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:49.946533 32205 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:11:49.997947 32163 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.001797 31999 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.947415 32262 consensus_queue.cc:1059] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
W20260709 13:11:50.057299 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:50.058352 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 63124us
I20260709 13:11:50.064532 32263 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:50.056057 32163 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.059827 31999 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.126950 31999 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:50.128867 32250 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:50.129817 32168 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:50.076680 32092 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.130856 32092 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.134099 32092 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:50.139326 32232 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.141796 32232 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:50.142848 32237 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 13:11:50.069489 32023 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.099763 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:50.100339 32262 raft_consensus.cc:3055] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:50.061357 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:50.101078 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 87315.5us
W20260709 13:11:50.154567 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 93327.8us
I20260709 13:11:50.101930 32215 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:50.102207 32043 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:50.103109 32205 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:50.104943 32263 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:50.055230 32265 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.015s
W20260709 13:11:50.119562 32244 log.cc:927] Time spent T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.293s	user 0.000s	sys 0.000s
W20260709 13:11:50.120980 32249 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:11:50.124565 32203 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:50.130347 32253 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.180573 32265 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:50.188441 32268 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:50.154260 32262 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:50.188844 32263 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.192102 32203 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:50.202888 32203 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.204317 32203 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:50.208455 32201 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:50.212769 32240 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.388s	user 0.001s	sys 0.000s
I20260709 13:11:50.214205 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1339) took 1676 ms. Trace:
I20260709 13:11:50.214602 29080 rpcz_store.cc:276] 0709 13:11:48.537545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.587229 (+ 49684us) service_pool.cc:224] Handling call
0709 13:11:48.587556 (+   327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:48.655013 (+ 67457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:48.655059 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:48.685922 (+ 30863us) log.cc:844] Serialized 52 byte log entry
0709 13:11:49.587568 (+901646us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:49.621227 (+ 33659us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.894290 (+273063us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b1800063040. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:11:49.946870 (+ 52580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.946966 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.172587 (+225621us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:50.213966 (+ 41379us) raft_consensus.cc:1685] finished
0709 13:11:50.213989 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.214140 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46313,"prepare.queue_time_us":32177,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":363366,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":13515,"spinlock_wait_cycles":27904256,"thread_start_us":920659,"threads_started":2,"wal-append.queue_time_us":773984}
I20260709 13:11:50.166697 32043 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:50.218580 32236 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:50.218129 32043 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.192229 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:50.172158 32205 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:50.194316 29002 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:50.157582 32215 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:50.167516 32200 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:50.221189 29714 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:50.193197 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.228423 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.195801 32268 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:50.232640 29010 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:50.219897 32043 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:50.237247 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.237964 29008 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:50.238112 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:50.228432 32215 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:50.238747 29080 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:50.226326 32118 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:50.239498 29084 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:50.240553 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2186) took 2607 ms. Trace:
W20260709 13:11:50.242847 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 143195us
I20260709 13:11:50.234999 32200 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:50.232631 32265 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:50.251166 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:50.241771 32151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.251s	user 0.056s	sys 0.099s
I20260709 13:11:50.239904 32258 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:50.242069 32205 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.240120 32099 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:50.260663 32200 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.249573 32262 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.264104 32178 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:50.287425 32178 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:50.288486 32178 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.289235 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47384 (request call id 670) took 1751 ms. Trace:
I20260709 13:11:50.289565 32178 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:50.289655 29708 rpcz_store.cc:276] 0709 13:11:48.538008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.538179 (+   171us) service_pool.cc:224] Handling call
0709 13:11:50.289051 (+1750872us) spinlock_profiling.cc:243] Waited 9.42 ms on lock 0x7b5400650ed4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:50.289171 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21660928}
W20260709 13:11:50.270706 29712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47384 (request call id 668) took 2425 ms (client timeout 1527 ms). Trace:
W20260709 13:11:50.290772 29712 rpcz_store.cc:269] 0709 13:11:47.841947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.842217 (+   270us) service_pool.cc:224] Handling call
0709 13:11:50.254699 (+2412482us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540021f554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:50.266848 (+ 12149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4759342848}
W20260709 13:11:50.292644 29709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 430) took 1749 ms (client timeout 2092 ms). Trace:
I20260709 13:11:50.292794 32265 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:50.293133 29709 rpcz_store.cc:269] 0709 13:11:48.543570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.555099 (+ 11529us) service_pool.cc:224] Handling call
0709 13:11:50.292436 (+1737337us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b5400650254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:50.292574 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206764288}
I20260709 13:11:50.295343 32265 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:50.295954 29082 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:11:50.292757 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 96208.5us
I20260709 13:11:50.298870 32232 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:50.299031 31960 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:50.299496 32232 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.299643 31960 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.300283 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.300886 29078 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 4.
I20260709 13:11:50.301867 32232 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:50.271250 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.303097 32025 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.273145 32205 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:50.303051 32024 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:50.303857 32025 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.304131 32024 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.306348 32025 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.306447 32024 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.307065 32262 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:50.307556 32024 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:50.269843 32151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.283s	user 0.057s	sys 0.100s
I20260709 13:11:50.308336 31529 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.308109 32024 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.308918 31529 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.310325 32024 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.311247 31529 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.271585 32258 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.274125 32268 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.319100 32268 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:50.318938 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:50.320819 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.322950 32224 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:50.323498 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:50.324491 31975 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:50.324952 31975 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.301927 31960 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:50.325476 31975 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:50.325946 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:50.326493 29081 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:50.276927 31823 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:50.277525 32099 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.303247 32232 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:50.244292 29154 rpcz_store.cc:276] 0709 13:11:47.633129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.633268 (+   139us) service_pool.cc:224] Handling call
0709 13:11:47.830394 (+197126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:47.830782 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:47.830822 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:47.845008 (+ 14186us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.193920 (+1348912us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b18003181c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:11:49.592784 (+398864us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:49.592863 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.593035 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.120405 (+527370us) spinlock_profiling.cc:243] Waited 7.76 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:50.240302 (+119897us) raft_consensus.cc:1685] finished
0709 13:11:50.240327 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.240486 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2252,"raft.queue_time_us":88668,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":17509,"spinlock_wait_cycles":24459904,"thread_start_us":398351,"threads_started":1,"wal-append.queue_time_us":1856135}
I20260709 13:11:50.314134 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:50.316710 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:50.336591 29712 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:50.313850 32258 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:50.340451 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:50.342733 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.342846 29715 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:50.248898 32118 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.341151 32232 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.358276 32232 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:50.359586 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 87400.3us
W20260709 13:11:50.346088 32148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.972s	user 0.015s	sys 0.015s
W20260709 13:11:50.365376 32148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.286s	user 0.016s	sys 0.016s
I20260709 13:11:50.377314 32099 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:50.378618 32236 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.380074 32236 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:50.381349 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.382395 32236 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:50.382859 32118 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:50.383186 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.383862 29154 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 3.
I20260709 13:11:50.385207 29159 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:50.386014 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.386839 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:50.387354 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:50.387377 29159 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:11:50.387529 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.388319 29152 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:50.391449 32163 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.387552 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.393085 29151 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:11:50.385890 32222 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.396072 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.396654 29151 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:50.398821 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.396318 29155 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:50.399494 29151 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:50.400691 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.406635 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:50.401538 29155 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:11:50.408619 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.409493 29153 raft_consensus.cc:2410] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:11:50.410291 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:50.410846 29150 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:50.413419 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.414058 29158 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:50.414916 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.415603 29157 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:50.418120 29159 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
W20260709 13:11:50.451048 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 85278.8us
I20260709 13:11:50.408387 32222 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:50.457868 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:50.511391 29081 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:50.511556 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.465660 29003 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:50.465962 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.472743 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.513845 29010 raft_consensus.cc:2410] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:11:50.469213 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 949368us
I20260709 13:11:50.514006 29008 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:50.479642 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.515329 29083 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:50.480929 29084 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:50.488000 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.516788 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:50.481314 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.512247 29004 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:50.481690 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.488158 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:50.481863 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:50.519852 29085 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:50.517642 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.465627 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2156) took 1615 ms. Trace:
W20260709 13:11:50.521749 29125 connection.cc:256] RPC call timeout handler was delayed by -1.1058414609999545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 13:11:50.480370 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.508546 29712 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:11:50.522284 29125 connection.cc:256] RPC call timeout handler was delayed by -1.0311414610000611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 13:11:50.522861 29078 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:50.504144 29080 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:50.521317 29002 rpcz_store.cc:276] 0709 13:11:48.850588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.694058 (+843470us) service_pool.cc:224] Handling call
0709 13:11:49.699870 (+  5812us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.225859 (+525989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.465444 (+239585us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54002053d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:50.465583 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1134790400}
I20260709 13:11:50.491991 28905 catalog_manager.cc:5697] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:50.512866 29683 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:50.525686 32148 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:50.526255 32148 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.486763 32210 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.527976 32210 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:50.528126 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:50.528606 32210 raft_consensus.cc:697] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:50.528901 29083 raft_consensus.cc:2368] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8.
I20260709 13:11:50.528895 32025 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.530005 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.529629 32210 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.529824 32025 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:50.460323 32275 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:11:50.531519 31976 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:50.532662 32025 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.534333 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:50.530685 29083 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:50.536286 32025 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:50.536468 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:50.537091 32203 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.510263 29150 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:50.537847 32203 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:50.510365 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.535864 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:50.531950 31976 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.471887 29051 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:50.539995 32203 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.540756 31976 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:50.545305 31629 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:50.547710 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 95786.6us
W20260709 13:11:50.541375 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 69551.2us
I20260709 13:11:50.548561 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:50.549640 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2170) took 1193 ms. Trace:
I20260709 13:11:50.541689 32099 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:50.550304 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:50.551154 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.551618 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:50.569473 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.537055 32025 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:50.552268 32235 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:50.552804 32043 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:50.558209 32024 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:50.564849 31974 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:50.566053 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:50.705778 29048 leader_election.cc:336] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.847s (SENT)
W20260709 13:11:50.600181 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.553603 32023 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:50.550026 29001 rpcz_store.cc:276] 0709 13:11:49.356345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.754235 (+397890us) service_pool.cc:224] Handling call
0709 13:11:49.754604 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:49.906298 (+151694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.549393 (+643095us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540035d9d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:50.549584 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327796608}
I20260709 13:11:50.550328 32099 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.541884 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:50.750092 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 208303us
I20260709 13:11:50.527014 28905 catalog_manager.cc:5697] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 4 to 2, term changed from 4 to 3, leader changed from <none> to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65), ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from VOTER to NON_VOTER, now has a pending config: VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65), VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66), VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67), VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:50.569499 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.667626 31976 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.518765 29082 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:50.575212 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:50.601850 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 52116.9us
W20260709 13:11:50.596863 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.764868 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:50.459539 31910 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:50.704479 32043 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:50.589286 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:50.466365 32222 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.562786 32230 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:50.786111 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:50.787586 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 81854.2us
I20260709 13:11:50.789014 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.789934 29077 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:50.784799 32168 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:50.793344 32168 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.793787 32168 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:50.794350 32215 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.002s
I20260709 13:11:50.794777 32215 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:50.796486 29048 leader_election.cc:336] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.075s (SENT)
I20260709 13:11:50.629812 29084 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:50.797108 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:50.634068 29150 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:50.635345 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.640508 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.543169 32186 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:50.649003 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:50.801738 29714 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:50.649019 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.666231 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:50.668447 28906 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52676 (request call id 243) took 22220 ms (client timeout 15000 ms). Trace:
I20260709 13:11:50.804585 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.670475 32148 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:50.670804 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.643353 32231 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:50.671295 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.458760 31890 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:50.693241 29080 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:50.735108 31974 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:50.704175 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 152662us
W20260709 13:11:50.711309 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.748915 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.550459 32203 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:50.751829 29083 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:11:50.814283 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 256430us
I20260709 13:11:50.704125 32025 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:50.461299 31975 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:50.781111 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 191937us
I20260709 13:11:50.750336 32099 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:50.796602 32276 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:11:50.799757 29078 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:50.800318 29081 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:50.805339 29084 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:50.780803 32043 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:50.775513 31910 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:50.808722 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.806811 32200 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:50.788473 32222 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:50.815753 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.753015 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.817760 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:50.821261 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:50.828233 29048 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.132s (SENT)
I20260709 13:11:50.891214 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:50.891430 29048 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:50.817888 31529 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:50.830358 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:50.882525 29077 raft_consensus.cc:2410] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:50.812300 31890 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:50.890298 29007 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:50.941299 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.941884 29082 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:50.892511 29151 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:50.609313 29085 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:11:50.627279 29153 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:50.896435 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:50.904479 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 76260.4us
I20260709 13:11:50.885545 32234 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:50.902885 32253 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:50.914816 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:50.950691 29005 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:50.827457 32203 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:50.901651 31921 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:50.938205 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.860577 32231 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.944341 29159 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 5.
I20260709 13:11:50.960757 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.961370 29159 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:50.962167 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:50.948251 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 48691.8us
I20260709 13:11:50.949127 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:50.949396 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.951020 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.952728 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:50.963181 29007 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:50.963647 29153 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:50.969810 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 218947us
I20260709 13:11:50.970465 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:50.971331 29004 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:50.979532 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.981195 29125 leader_election.cc:400] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6.
I20260709 13:11:50.981593 29125 leader_election.cc:403] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:50.969144 28905 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 3 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:50.990387 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:50.993050 29159 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:11:51.027102 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.027912 29158 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 3.
I20260709 13:11:51.025295 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.030617 29004 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:50.958097 32234 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.035277 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.035597 32234 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:50.968456 32148 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:51.036407 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.037031 29151 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:51.036875 32148 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.039156 32148 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:51.039762 29151 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:11:51.048120 32178 consensus_meta.cc:287] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.758s	user 0.012s	sys 0.002s
W20260709 13:11:50.803540 28906 rpcz_store.cc:269] 0709 13:11:28.441699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.441944 (+   245us) service_pool.cc:224] Handling call
0709 13:11:50.652193 (+22210249us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:38.635207 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:38.635888 (+   681us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:38.635943 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:11:38.635955 (+    12us) tablet.cc:662] Decoding operations
0709 13:11:38.637780 (+  1825us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:38.637895 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:38.637966 (+    71us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:11:38.642083 (+  4117us) tablet.cc:701] Row locks acquired
0709 13:11:38.642132 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:11:38.642331 (+   199us) write_op.cc:270] Start()
0709 13:11:38.642483 (+   152us) write_op.cc:276] Timestamp: P: 1783602698642262 usec, L: 0
0709 13:11:38.642503 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:11:38.685946 (+ 43443us) log.cc:844] Serialized 9363 byte log entry
0709 13:11:39.925399 (+1239453us) op_driver.cc:464] REPLICATION: finished
0709 13:11:40.387246 (+461847us) write_op.cc:301] APPLY: starting
0709 13:11:40.387377 (+   131us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:40.388150 (+   773us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:40.388343 (+   193us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:40.391970 (+  3627us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:40.671057 (+279087us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 13:11:40.684826 (+ 13769us) write_op.cc:312] APPLY: finished
0709 13:11:40.901420 (+216594us) log.cc:844] Serialized 164 byte log entry
0709 13:11:41.115329 (+213909us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:41.115585 (+   256us) write_op.cc:454] Released schema lock
0709 13:11:41.115690 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:41.118902 (+  3212us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:41.118933 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27597,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":443904,"apply.run_cpu_time_us":16272,"apply.run_wall_time_us":731641,"num_ops":24,"prepare.queue_time_us":178328,"prepare.run_cpu_time_us":44576,"prepare.run_wall_time_us":266708,"raft.queue_time_us":107811,"raft.run_cpu_time_us":54867,"raft.run_wall_time_us":446040,"replication_time_us":1258412,"thread_start_us":839637,"threads_started":5,"wal-append.queue_time_us":1156498,"wal-append.run_cpu_time_us":72367,"wal-append.run_wall_time_us":2314874}]]}
I20260709 13:11:51.049830 29151 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
W20260709 13:11:51.052187 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 59690.2us
I20260709 13:11:51.045042 29009 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:51.046535 32203 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:51.054179 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:51.051882 29159 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:50.998171 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.095013 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.095840 29158 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:51.026050 28905 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:11:51.057471 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 229065us
W20260709 13:11:51.109225 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 55248us
I20260709 13:11:50.978341 32098 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.096231 29005 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:51.110146 29125 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:51.026923 32202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.010s	sys 0.009s
I20260709 13:11:51.135865 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:51.140332 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:50.996829 32222 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6.
W20260709 13:11:51.148468 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.148943 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.056386 32238 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.160768 29004 raft_consensus.cc:2410] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:50.998554 31976 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:51.155336 29007 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:11:51.097075 32234 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.157675 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.115918 32178 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:50.978300 32231 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:51.155987 32202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.010s	sys 0.012s
I20260709 13:11:51.163403 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:11:51.179924 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.161458 32284 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:51.162407 31975 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.161324 32023 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:51.137567 32098 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.203828 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.204655 29125 leader_election.cc:400] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8.
I20260709 13:11:51.204967 29125 leader_election.cc:403] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:11:51.210889 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.207087 32288 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:11:51.190181 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 54427.3us
I20260709 13:11:50.968216 31910 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.184163 28905 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 2 to 3, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:51.220479 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.221089 29007 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:51.193224 29010 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:11:51.196539 29158 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:51.189929 32178 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:51.207978 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:51.211928 29159 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:51.198700 32284 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:51.224552 29712 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.013s	sys 0.029s
W20260709 13:11:51.218976 29009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2142) took 2565 ms (client timeout 1667 ms). Trace:
W20260709 13:11:51.225113 29009 rpcz_store.cc:269] 0709 13:11:48.648764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.648928 (+   164us) service_pool.cc:224] Handling call
0709 13:11:51.213813 (+2564885us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b5400228654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:51.213951 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5440516864}
W20260709 13:11:51.224871 29712 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.013s	sys 0.029s
I20260709 13:11:51.195013 32234 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.188985 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.234977 29153 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:51.209684 32222 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8.
I20260709 13:11:51.231566 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:51.229215 32284 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:51.262899 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:51.259856 32241 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.023s	sys 0.029s
I20260709 13:11:51.271915 32186 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:51.278635 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.223775 32288 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:51.274821 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:51.229396 32178 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:51.276033 32241 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.023s	sys 0.029s
W20260709 13:11:51.282513 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 59540.5us
I20260709 13:11:51.168872 32238 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.286212 32200 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:51.288321 32023 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:51.288653 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:51.303102 29153 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:11:51.288928 28905 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 2 to 3, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 3 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:51.290688 29151 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:11:51.291455 32286 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:11:51.311633 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:51.292315 32280 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:11:51.285442 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:51.313990 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:51.314814 29050 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:51.314926 32268 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:51.315414 32268 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.277134 31529 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:11:51.238584 32282 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:51.298372 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:51.276572 31910 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:51.359640 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61379.5us
I20260709 13:11:51.361240 29682 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:51.362514 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:51.322101 28976 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:51.373893 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 51861.5us
I20260709 13:11:51.344784 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.315361 32043 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.315898 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:51.378734 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 62887.9us
I20260709 13:11:51.293293 32186 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:51.326625 32273 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.874s	user 0.004s	sys 0.000s
I20260709 13:11:51.384224 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:51.315569 32118 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:51.323060 32202 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:51.382509 29001 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
W20260709 13:11:51.377688 32168 consensus_meta.cc:287] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.004s
I20260709 13:11:51.360337 32222 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:51.400251 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.382284 32212 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:51.318018 31921 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:51.300981 32025 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.315956 32268 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:51.412451 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.352931 32098 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.394747 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:51.363003 32282 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:51.477458 32215 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.477635 32282 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:51.477962 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 83305.5us
I20260709 13:11:51.401952 32168 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.479173 29682 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:51.395274 31629 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:51.378108 32023 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:51.378269 32253 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:51.494124 32253 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.494238 32212 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:51.382512 32209 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:51.494298 32098 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:51.393697 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:51.494825 32209 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:51.382592 32186 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:51.494560 32253 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:51.407467 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 240539us
I20260709 13:11:51.409073 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:51.500156 32231 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.501005 32231 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:51.501549 32231 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:51.502214 29159 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:51.503042 31529 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.503518 31529 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.504958 32253 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.505731 31529 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:51.392570 32118 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.506778 32118 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:51.506913 32253 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.508669 32025 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:51.510167 32234 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:51.317337 31974 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.396757 32202 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:51.352728 32284 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.427124 32203 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:51.436537 29683 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:51.513360 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 76958.3us
I20260709 13:11:51.400346 32222 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.438110 28905 catalog_manager.cc:5697] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 4, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 4 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:51.444416 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:51.517496 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 73193.1us
I20260709 13:11:51.519428 29077 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:51.403997 32024 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:51.446460 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.522782 29715 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:51.522861 29005 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:51.407990 31921 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:51.452108 29154 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:51.453225 32043 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:51.439774 32185 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:51.535128 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:51.457844 32099 consensus_meta.cc:287] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.002s
I20260709 13:11:51.319967 31890 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.462268 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:51.536598 29083 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:51.463001 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:51.537410 29082 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:51.463469 32238 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.465663 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.468364 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:51.476763 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1708) took 1238 ms. Trace:
I20260709 13:11:51.546554 29079 rpcz_store.cc:276] 0709 13:11:50.238706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.243955 (+  5249us) service_pool.cc:224] Handling call
0709 13:11:50.251119 (+  7164us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.252375 (+  1256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.252406 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:50.252847 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:11:50.341861 (+ 89014us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:50.341919 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.342022 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:51.326055 (+984033us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:51.476452 (+150397us) raft_consensus.cc:1685] finished
0709 13:11:51.476474 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:51.476703 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":944,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":3353344,"thread_start_us":88574,"threads_started":1,"wal-append.queue_time_us":112136}
I20260709 13:11:51.481133 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:51.446708 29153 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:51.485121 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.392685 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.495288 32209 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:51.427024 29152 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:51.514590 29122 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:51.552140 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:51.551819 28905 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 4, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 4 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:51.502533 32231 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.554064 32234 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:51.521505 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.558619 29007 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:51.522274 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:51.479938 32168 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:51.527745 29150 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:11:51.535933 29085 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:11:51.552764 29081 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:11:51.552654 29122 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:51.520814 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:51.556238 29714 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:11:51.518083 32222 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:51.522631 31976 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:51.522395 32284 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:51.534341 31921 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:51.580229 28905 catalog_manager.cc:5697] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:51.445894 32288 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.584167 31976 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.584594 31976 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:51.553784 32099 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.588424 29078 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 5: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:51.588598 32178 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.589320 32099 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:51.591239 32178 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:51.571189 31890 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:51.591028 29683 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:51.604846 29683 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:51.572328 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 91257.1us
I20260709 13:11:51.606459 29682 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:51.553651 32185 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.576084 32118 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:51.556684 32214 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.032s	sys 0.043s
I20260709 13:11:51.577319 32251 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:51.580169 32215 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:51.556208 31529 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:51.590955 32202 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.563756 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:51.562450 32183 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:51.639114 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1458) took 1115 ms. Trace:
I20260709 13:11:51.571820 32234 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:51.639609 29152 rpcz_store.cc:276] 0709 13:11:50.523454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.540404 (+ 16950us) service_pool.cc:224] Handling call
0709 13:11:50.540785 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:51.552144 (+1011359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.638832 (+ 86688us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54002b66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:51.639042 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037419776}
W20260709 13:11:51.649132 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.570087 29085 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:51.650468 29683 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:51.626281 32214 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.033s	sys 0.044s
W20260709 13:11:51.636775 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 73116.6us
I20260709 13:11:51.654973 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:51.637975 32025 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:51.644277 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:51.620234 32212 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:51.612294 29083 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:11:51.570703 32224 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:11:51.617458 32259 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:51.651512 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 376746us
I20260709 13:11:51.626505 32251 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.636232 31529 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.674952 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:51.616863 32262 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:51.673889 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:51.638707 32183 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:51.648705 32023 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:51.678442 32262 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:51.684825 29155 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.013s	sys 0.011s
I20260709 13:11:51.670425 32212 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:51.690282 29155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.013s	sys 0.011s
I20260709 13:11:51.652803 31921 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.675549 32178 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:51.674177 32259 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:51.677939 31529 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:51.675510 32251 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:51.716739 32303 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:51.677600 32284 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:11:51.764747 32284 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:51.765265 32284 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:51.766134 32284 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:51.768757 32299 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:11:51.770648 32209 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:51.717094 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 423878us
I20260709 13:11:51.773545 31921 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:51.773804 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.660496 31976 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.774747 28975 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:51.775483 32023 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:51.775959 32023 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:51.776726 32298 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:11:51.776889 32023 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.687017 32183 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:51.776940 31976 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:51.783859 29048 leader_election.cc:336] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 3.346s (SENT)
I20260709 13:11:51.675978 32253 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:11:51.691401 32210 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:51.702811 32262 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:51.768481 31529 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.716226 32212 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:51.812510 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:51.816880 29006 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:51.779096 32259 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:51.745584 32178 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:51.835457 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51641.6us
I20260709 13:11:51.788151 32168 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.839304 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:51.840552 29004 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:51.837705 32251 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.795604 32230 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:51.849025 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:51.914742 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65816.7us
I20260709 13:11:51.861254 32043 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:51.862596 32118 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:51.868731 32302 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:11:51.839375 32282 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.784248 32186 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.869882 32168 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.871485 32209 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.871975 32185 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:51.783349 32234 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.888697 32025 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:51.892562 32203 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:51.935956 32300 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:51.905697 32202 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:51.912148 29050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.006s	sys 0.040s
W20260709 13:11:51.923089 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.954547 31629 raft_consensus.cc:697] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:51.839309 32178 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:51.945298 29004 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:51.842967 32210 raft_consensus.cc:697] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:51.964921 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 571219us
W20260709 13:11:51.896595 29048 leader_election.cc:336] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 3.428s (SENT)
W20260709 13:11:51.977268 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 404808us
I20260709 13:11:51.981325 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.987576 29153 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:51.988574 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:51.988919 29712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 469) took 1320 ms. Trace:
I20260709 13:11:51.982227 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:51.722198 32268 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:51.973572 32203 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:51.946564 32024 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:51.935642 32230 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.990850 29150 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:51.991418 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:51.984164 29050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.006s	sys 0.040s
I20260709 13:11:51.996461 32288 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:51.984200 31975 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:51.997047 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:52.045158 29714 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:51.997015 31629 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.997978 29078 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:51.998380 29085 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:52.005350 32251 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:52.008550 32314 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.012879 32282 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:52.014927 31529 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:52.014948 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.019028 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:52.062762 29081 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:52.062768 29715 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:51.973801 32300 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:52.063803 32300 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:52.022302 29006 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:52.022953 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:52.065342 29713 raft_consensus.cc:2368] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9.
I20260709 13:11:52.065577 32168 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:52.022966 32215 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:11:52.026472 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 647016us
I20260709 13:11:51.996785 32210 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:51.989259 29712 rpcz_store.cc:276] 0709 13:11:50.663638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.663795 (+   157us) service_pool.cc:224] Handling call
0709 13:11:50.664173 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.664414 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:51.227946 (+563532us) log.cc:844] Serialized 14 byte log entry
0709 13:11:51.983559 (+755613us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:51.983609 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:51.983652 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.983717 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:51.983832 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37446,"prepare.queue_time_us":743644,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":38053,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":8396,"thread_start_us":1297128,"threads_started":3,"wal-append.queue_time_us":752148}
I20260709 13:11:52.024672 32024 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:52.059790 32314 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.041441 32288 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:52.070174 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 173612us
W20260709 13:11:52.075987 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 425544us
I20260709 13:11:52.077548 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1467) took 1276 ms. Trace:
I20260709 13:11:52.041532 32186 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:52.061554 32296 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.371s	user 0.000s	sys 0.003s
I20260709 13:11:52.080914 29077 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:52.081633 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:52.077932 29156 rpcz_store.cc:276] 0709 13:11:50.801202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.036279 (+235077us) service_pool.cc:224] Handling call
0709 13:11:51.036606 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:51.036952 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:51.036981 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:51.037448 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:11:51.399960 (+362512us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:51.400002 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.401265 (+  1263us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:52.077344 (+676079us) raft_consensus.cc:1685] finished
0709 13:11:52.077362 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:52.077484 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2772,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":362121,"threads_started":1,"wal-append.queue_time_us":619455}
I20260709 13:11:52.011822 32183 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:52.102954 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.103664 28976 connection.cc:256] RPC call timeout handler was delayed by -2.3301870369999733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 13:11:52.069288 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:52.086093 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:52.086200 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:52.107050 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:52.056928 32118 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:52.107970 28976 connection.cc:256] RPC call timeout handler was delayed by -2.2770870370000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 13:11:52.112488 28976 connection.cc:256] RPC call timeout handler was delayed by -2.2473870370000668s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 13:11:52.111889 29085 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
W20260709 13:11:52.112910 28976 connection.cc:256] RPC call timeout handler was delayed by -2.1501870370000233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
I20260709 13:11:52.113323 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1463) took 1449 ms. Trace:
W20260709 13:11:52.113515 28976 connection.cc:256] RPC call timeout handler was delayed by -2.1483870370000204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
W20260709 13:11:52.113945 28976 connection.cc:256] RPC call timeout handler was delayed by -2.0583870369999886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 13:11:52.114430 28976 connection.cc:256] RPC call timeout handler was delayed by -2.0034870370000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
W20260709 13:11:52.114817 28976 connection.cc:256] RPC call timeout handler was delayed by -1.9395870370000239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
I20260709 13:11:52.111852 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.112190 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.115748 29079 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:11:52.116496 29078 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:52.112821 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.118515 29081 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:52.115322 28976 connection.cc:256] RPC call timeout handler was delayed by -1.917987036999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
W20260709 13:11:52.121186 28976 connection.cc:256] RPC call timeout handler was delayed by -1.8720870370000284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
W20260709 13:11:52.121510 28976 connection.cc:256] RPC call timeout handler was delayed by -1.8153870370000504s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
W20260709 13:11:52.121917 28976 connection.cc:256] RPC call timeout handler was delayed by -1.8135870370000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
W20260709 13:11:52.122243 28976 connection.cc:256] RPC call timeout handler was delayed by -1.7343870370000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2182s for a response
W20260709 13:11:52.122584 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6875870370000712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
W20260709 13:11:52.122959 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6857870370000683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 13:11:52.123313 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6290870369999766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2299s for a response
W20260709 13:11:52.123677 28976 connection.cc:256] RPC call timeout handler was delayed by -1.5453870370000686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2392s for a response
I20260709 13:11:52.124650 29713 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:52.127022 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.129168 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.129936 29077 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:52.130810 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:52.132545 29713 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:52.137581 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 462626us
I20260709 13:11:52.140594 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:52.126294 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 287780us
I20260709 13:11:52.100353 32168 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.145218 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:52.113772 29155 rpcz_store.cc:276] 0709 13:11:50.664273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.664467 (+   194us) service_pool.cc:224] Handling call
0709 13:11:50.664877 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.665155 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:51.156825 (+491670us) log.cc:844] Serialized 14 byte log entry
0709 13:11:52.111919 (+955094us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:52.111978 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:52.112027 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.113148 (+  1121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:52.113268 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99660,"prepare.queue_time_us":367657,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":194743,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":1328171,"threads_started":3,"wal-append.queue_time_us":816369}
W20260709 13:11:52.146545 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 502283us
I20260709 13:11:52.150072 29682 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:52.113970 32148 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:52.150741 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:52.115278 32311 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:52.116674 32315 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.158478 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:52.146236 32241 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:52.186084 28976 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:52.176301 29078 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
W20260709 13:11:52.178318 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 462201us
I20260709 13:11:52.148824 32317 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.153326 32168 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.153787 32186 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:52.155086 31921 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:52.162609 32253 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:11:52.240185 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 78780.6us
I20260709 13:11:52.210808 29081 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:52.253924 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:52.262702 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 84691.9us
W20260709 13:11:52.263214 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44826.3us
I20260709 13:11:52.166042 32311 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:11:52.268877 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 191818us
W20260709 13:11:52.270668 29048 leader_election.cc:336] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.527s (SENT)
W20260709 13:11:52.271607 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 112239us
I20260709 13:11:52.148885 32169 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:11:52.155711 32231 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
W20260709 13:11:52.246553 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60577us
I20260709 13:11:52.271184 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:52.282641 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:52.283022 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:52.281286 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 130610us
I20260709 13:11:52.184837 32241 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:52.330662 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 68590.7us
W20260709 13:11:52.416168 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 151546us
I20260709 13:11:52.424111 31910 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:52.427443 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:52.428144 29006 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
W20260709 13:11:52.428665 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 270229us
I20260709 13:11:52.429152 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:52.429955 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:52.429185 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.551s (SENT)
I20260709 13:11:52.432463 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:52.428680 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:52.431955 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:52.436703 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:52.437471 29001 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:52.274503 32311 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:52.518699 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 236096us
W20260709 13:11:52.423241 32262 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.004s
I20260709 13:11:52.242190 32327 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:52.267831 32224 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:52.543561 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.416287 32241 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:52.243074 32317 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:52.183851 32023 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.335s
I20260709 13:11:52.569727 29001 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:52.280006 32253 raft_consensus.cc:697] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:52.272430 32259 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.583916 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:52.585878 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:52.273109 32209 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:52.551509 32308 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:52.162134 32288 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.598951 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:52.187723 32315 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:52.549253 32268 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:52.603981 29714 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:11:52.611507 32314 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:52.612447 32314 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:52.613173 32259 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:52.612991 32314 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.493103 32326 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:11:52.627396 29712 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:52.595958 32340 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:52.630388 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:11:52.637024 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.627930 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 342539us
I20260709 13:11:52.601588 32308 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.604547 32230 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:52.570763 32262 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.648717 32348 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:52.600435 32327 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:52.653733 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:11:52.653867 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 370987us
I20260709 13:11:52.614553 32183 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:52.642827 31529 consensus_queue.cc:1175] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ca45420e033847489ad989c1c6eed3ef" responder_term: 6 status { last_received { term: 6 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 6 index: 8 } }
I20260709 13:11:52.643831 32340 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:52.682888 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:52.652555 32202 consensus_meta.cc:287] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.000s
I20260709 13:11:52.653208 32308 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:52.655737 32326 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:11:52.673044 32203 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.000s
I20260709 13:11:52.663192 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ed915f0be134d71b36d8fc8f7b5830b with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:34914
I20260709 13:11:52.703519 32326 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:52.696561 32268 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:52.704429 32314 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:52.704613 32326 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.704967 32203 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:52.706067 32326 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:52.706904 32307 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.698s	user 0.007s	sys 0.007s
I20260709 13:11:52.658102 32327 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:52.697782 32168 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:52.710367 29122 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:52.710558 32288 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:52.711843 32025 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.712374 32025 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.712651 32340 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:52.714003 32340 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:52.714433 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:52.717278 29008 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:52.718281 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:52.718495 32214 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:52.719029 32214 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.719460 32214 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:52.662681 32343 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.720520 32343 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:52.722922 32343 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:52.673139 32178 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.001s
I20260709 13:11:52.665640 32317 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:52.700095 32262 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:52.677865 32329 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 13:11:52.663417 32330 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:11:52.679102 32336 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:11:52.677320 32222 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.065s	user 0.009s	sys 0.002s
I20260709 13:11:52.707235 32203 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:52.696800 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 267614us
I20260709 13:11:52.677512 32315 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:52.714542 32025 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:52.611554 32253 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:52.733517 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.562s (SENT) [suppressed 32 similar messages]
I20260709 13:11:52.730399 29001 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:11:52.744477 29048 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.605s (SENT)
I20260709 13:11:52.746183 29122 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:52.748560 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 65785.8us
W20260709 13:11:52.749210 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 332499us
I20260709 13:11:52.654866 31921 consensus_queue.cc:1059] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:11:52.764071 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:52.755332 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.791594 32314 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:52.792147 32314 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.789592 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:52.792573 32314 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:52.678432 32212 consensus_meta.cc:287] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.835s	user 0.008s	sys 0.001s
I20260709 13:11:52.707717 32327 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:52.756409 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:52.805552 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:52.806321 29155 raft_consensus.cc:2376] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7. Re-sending same reply.
W20260709 13:11:52.736328 32316 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.551s	user 0.000s	sys 0.002s
I20260709 13:11:52.813190 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:52.814821 29078 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:11:52.812319 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:52.815972 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.816601 29080 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
W20260709 13:11:52.817467 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61190.5us
W20260709 13:11:52.807797 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 162102us
I20260709 13:11:52.809393 29154 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
W20260709 13:11:52.809504 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65052.1us
I20260709 13:11:52.751039 32315 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:52.809525 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:52.819437 29048 leader_election.cc:336] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.625s (SENT)
I20260709 13:11:52.795465 31529 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:52.820587 29082 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:11:52.828248 29082 pending_rounds.cc:85] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 3
I20260709 13:11:52.828485 29082 pending_rounds.cc:107] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:11:52.828717 29082 raft_consensus.cc:2889] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:52.831460 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:52.816771 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:52.832077 29157 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:52.832360 29150 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:52.818217 29683 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:52.823580 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.823580 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:52.824364 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:52.822616 28976 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:52.833822 29153 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:52.834674 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:52.831171 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:52.836004 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:52.839378 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:52.841455 29048 leader_election.cc:336] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.635s (SENT)
I20260709 13:11:52.841976 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:52.842119 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.842833 29683 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:52.826787 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.843562 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.847440 29159 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 5: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:52.822088 32346 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:52.733074 32222 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:52.806644 32341 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:52.727619 32178 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:52.749996 32202 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:11:52.858613 28976 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:52.848548 29007 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:11:52.852607 29076 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:52.852488 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.854233 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:52.861937 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.825801 32323 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:52.879920 29048 leader_election.cc:336] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.652s (SENT)
I20260709 13:11:52.881343 29084 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:52.882140 29085 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:52.881708 29005 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:52.826438 32339 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:52.809295 32023 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 13:11:52.886749 29048 leader_election.cc:336] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.667s (SENT)
I20260709 13:11:52.888238 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.813548 32212 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:52.885902 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46268.6us
I20260709 13:11:52.851686 32351 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.887801 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.883787 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:52.892513 29048 leader_election.cc:336] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.683s (SENT)
I20260709 13:11:52.852157 31529 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:52.853282 32098 raft_consensus.cc:2804] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:11:52.903860 29048 leader_election.cc:336] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.770s (SENT)
I20260709 13:11:52.902765 29158 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:52.903393 29083 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:52.905257 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.851791 32262 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:11:52.894994 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:52.916031 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:52.892077 32339 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.889178 32099 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.916877 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.901525 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:52.905342 29005 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:11:52.911854 29048 leader_election.cc:336] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.852s (SENT)
I20260709 13:11:52.917685 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:52.852353 32346 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:52.919507 32324 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:52.920125 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.920346 29078 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:52.921113 29048 leader_election.cc:336] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.881s (SENT)
W20260709 13:11:52.922353 32024 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.847s	user 0.010s	sys 0.000s
I20260709 13:11:52.905880 32315 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:52.922781 32024 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:52.924988 32337 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:52.921711 29048 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:52.883420 32345 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:11:52.929417 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 70890.8us
I20260709 13:11:52.889894 32025 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:11:52.928122 29048 leader_election.cc:336] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.916s (SENT)
I20260709 13:11:52.939054 32282 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:11:52.943642 32322 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:52.945268 32212 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:52.947738 32300 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.883s	user 0.006s	sys 0.003s
I20260709 13:11:52.948159 32300 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.963665 29708 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:52.943043 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:52.958515 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:52.859478 32336 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:52.936192 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:52.901883 32353 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:52.949376 32148 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:52.972172 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44116.5us
I20260709 13:11:52.972293 29005 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:52.970803 32354 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.971343 31910 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:52.927768 32098 raft_consensus.cc:697] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:52.853615 32347 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:52.928006 32339 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:52.933107 32337 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:53.007521 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71408.3us
I20260709 13:11:53.052505 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:53.024796 29048 leader_election.cc:336] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.692s (SENT)
I20260709 13:11:53.053596 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:53.057148 32098 consensus_queue.cc:237] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.892962 32323 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:52.992461 32202 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:53.060729 32202 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:53.061278 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.062461 32323 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:53.061865 32202 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.063617 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:52.935927 32234 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:11:53.061964 29153 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:52.980100 32209 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:52.910395 32351 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.085626 29079 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:52.992375 32341 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:52.935503 32099 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.088553 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:53.078833 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 54118.9us
I20260709 13:11:53.035861 32200 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.105608 29076 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:52.981261 32355 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:11:53.095911 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43535.4us
W20260709 13:11:53.106160 29048 leader_election.cc:336] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.989s (SENT)
I20260709 13:11:53.034612 32317 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:53.046125 32315 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:53.046252 32347 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.966383 32025 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.132625 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:53.063791 32202 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:53.049757 32353 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:53.051947 32212 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:53.160447 32212 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.146271 32347 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:53.162544 32212 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:53.091787 31910 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:53.110034 32360 tablet_replica.cc:333] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:11:53.134883 29714 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:53.135358 32350 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:11:53.136314 32178 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.136656 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.139192 32314 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.139685 29077 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:11:53.146561 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 149537us
I20260709 13:11:53.146816 29083 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:53.146510 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:53.133713 29048 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:53.147846 29076 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
W20260709 13:11:53.195333 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 89197.6us
I20260709 13:11:53.195572 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:53.196781 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:53.149529 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:53.152967 32356 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:53.146544 32148 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.157294 32099 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:53.157512 32332 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:11:53.158540 32300 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:53.126144 31529 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.111577 31629 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:53.171527 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:53.111901 32355 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:53.174243 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.197333 29076 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:53.196456 29048 leader_election.cc:336] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.136s (SENT)
I20260709 13:11:53.046789 32354 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:53.222520 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51096us
I20260709 13:11:53.235394 29125 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:53.235870 29076 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:53.225426 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.236596 29151 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:53.236550 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:53.226795 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:53.238204 32099 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:53.238355 32355 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:53.239353 29048 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.220s (SENT)
I20260709 13:11:53.239677 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:53.240365 32349 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:53.240531 29048 leader_election.cc:336] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.244s (SENT)
I20260709 13:11:53.240958 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:53.241658 32349 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:53.242180 29153 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
W20260709 13:11:53.243597 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 94151.3us
W20260709 13:11:53.196420 32311 consensus_meta.cc:287] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.000s
I20260709 13:11:53.246318 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:53.192930 32024 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.129724 32200 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.256371 32355 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:53.256731 32372 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:53.257000 32323 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:11:53.258145 32358 log.cc:927] Time spent T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:11:53.258375 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:53.258571 32353 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.217303 31910 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:53.259338 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.224382 32178 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:53.264782 32367 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:11:53.265595 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.243710 32241 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
I20260709 13:11:53.282500 29151 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:11:53.234581 32340 consensus_meta.cc:287] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.004s
I20260709 13:11:53.106098 32346 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.285154 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:53.285320 29083 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:53.170651 32317 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.255728 32315 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:53.334792 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:53.335950 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:53.337266 29085 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:11:53.337554 32308 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.597s	user 0.007s	sys 0.002s
I20260709 13:11:53.337826 29082 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:11:53.337982 32308 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.295297 29079 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:53.340799 32351 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:53.304096 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45730.8us
I20260709 13:11:53.313515 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:53.321205 32214 consensus_meta.cc:287] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.601s	user 0.003s	sys 0.007s
I20260709 13:11:53.314932 32025 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.325332 32314 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:53.309528 32372 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:11:53.327733 32359 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.423s	user 0.000s	sys 0.009s
I20260709 13:11:53.328418 32222 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.368294 32354 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.255234 32311 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.371011 32317 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:53.313587 32241 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.308065 32339 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.383391 32346 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:53.384243 32339 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.220961 32148 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.310715 32288 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:53.287269 32209 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.374150 29122 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:53.425173 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51116.7us
I20260709 13:11:53.392656 32315 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:53.375890 29077 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:11:53.342929 29006 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:11:53.379194 29079 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:53.379877 32214 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:53.387306 29158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1478) took 1406 ms (client timeout 1830 ms). Trace:
W20260709 13:11:53.430402 29158 rpcz_store.cc:269] 0709 13:11:51.980877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.981954 (+  1077us) service_pool.cc:224] Handling call
0709 13:11:53.387103 (+1405149us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b540013cc54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:53.387248 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2084730624}
I20260709 13:11:53.387900 29155 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:11:53.389564 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:53.331486 32369 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:11:53.412572 29081 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:53.412927 29715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 694) took 1140 ms. Trace:
I20260709 13:11:53.414122 29005 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:53.366094 32200 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:53.344841 29683 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:53.363967 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:53.387322 32268 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:11:53.447639 32222 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:11:53.444504 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 80656.7us
I20260709 13:11:53.448675 32148 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.449520 29125 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:53.420271 32347 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.453352 32347 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.454133 32222 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:53.455842 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:53.445590 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 100809us
I20260709 13:11:53.436962 29715 rpcz_store.cc:276] 0709 13:11:52.273041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.273283 (+   242us) service_pool.cc:224] Handling call
0709 13:11:52.273782 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:52.606191 (+332409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:52.606226 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:52.644343 (+ 38117us) log.cc:844] Serialized 26 byte log entry
0709 13:11:52.830331 (+185988us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:52.830364 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.830424 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:53.327127 (+496703us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:53.387464 (+ 60337us) raft_consensus.cc:1685] finished
0709 13:11:53.387486 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:53.412853 (+ 25367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23240,"raft.queue_time_us":139663,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":48825,"spinlock_wait_cycles":12702592,"thread_start_us":482779,"threads_started":2,"wal-append.queue_time_us":243504}
W20260709 13:11:53.436339 32327 consensus_meta.cc:287] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.000s
I20260709 13:11:53.437587 32322 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:53.457351 32300 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:11:53.488479 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 234760us
I20260709 13:11:53.459600 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:53.461815 32372 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:53.492821 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:53.458918 29683 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:53.500000 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:53.500705 29077 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:53.488521 32326 consensus_meta.cc:287] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.782s	user 0.009s	sys 0.005s
I20260709 13:11:53.504393 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:53.504590 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45072.1us
I20260709 13:11:53.463770 32327 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.506392 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:53.506492 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:53.462860 32148 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:11:53.470883 32308 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:53.509233 29080 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:11:53.532768 32351 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.533448 32351 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.534636 32323 consensus_queue.cc:1059] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:53.535950 32323 raft_consensus.cc:3055] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:53.536482 32323 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:53.532644 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:53.538802 29715 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:53.538524 32323 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.540150 32351 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:53.540350 32323 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:53.542294 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 102768us
I20260709 13:11:53.542330 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:53.545737 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.546561 29005 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:53.548584 29077 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:53.493839 32322 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:53.503011 32206 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:53.505342 32025 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:53.502378 32300 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:53.555799 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56677.6us
I20260709 13:11:53.554770 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:53.492949 32341 consensus_queue.cc:1059] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:11:53.524063 32354 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:53.574918 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:53.551900 32387 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:53.599128 29005 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:53.551865 32353 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:53.506254 32098 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:11:53.554067 32238 consensus_queue.cc:1059] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:11:53.552604 32311 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:53.611140 32376 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:11:53.597504 32025 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:53.752964 32025 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.755987 29002 raft_consensus.cc:3055] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:53.756556 29002 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:53.757421 29002 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.758792 29002 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:53.605580 32343 raft_consensus.cc:2764] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:11:53.607928 32353 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:53.760830 32353 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:53.624891 32378 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:11:53.725498 32268 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.765486 32268 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.726380 32354 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.768333 32354 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.770588 32354 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:53.573077 31976 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:53.771438 31976 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.549974 32326 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:11:53.727118 32178 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.727229 32372 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.727873 31529 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.785328 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.728755 32308 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:53.600553 32043 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:53.621340 32300 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.601490 32341 raft_consensus.cc:3055] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:53.736608 31629 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:53.737252 32023 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.737330 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1887) took 1024 ms. Trace:
I20260709 13:11:53.740686 32322 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.743220 32327 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:53.819540 29006 rpcz_store.cc:276] 0709 13:11:52.712755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.715239 (+  2484us) service_pool.cc:224] Handling call
0709 13:11:52.715568 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.429407 (+713839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.737099 (+307692us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b5400139554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:53.737272 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379514752}
I20260709 13:11:53.747262 32387 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:53.749909 32202 consensus_meta.cc:287] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.008s
I20260709 13:11:53.542558 32315 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:53.756019 29009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33750 (request call id 457) took 2235 ms (client timeout 1892 ms). Trace:
W20260709 13:11:53.832216 29009 rpcz_store.cc:269] 0709 13:11:51.520463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.520580 (+   117us) service_pool.cc:224] Handling call
0709 13:11:53.755707 (+2235127us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003e21d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:53.755917 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3043362560}
I20260709 13:11:53.777330 29154 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:53.778949 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.779876 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.833808 29001 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:53.787386 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.606331 29080 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:11:53.795724 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:53.836222 29151 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:53.797156 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:53.728633 32351 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:53.800585 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:53.800688 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 219035us
I20260709 13:11:53.839789 29152 raft_consensus.cc:3060] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:53.803591 32268 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:53.807624 29157 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:11:53.808696 29680 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:53.814049 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.843494 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.845201 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.845927 29009 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:53.806516 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:53.805003 32380 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.248s	user 0.007s	sys 0.003s
I20260709 13:11:53.827812 29159 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:53.787060 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.837121 29156 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:53.837867 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.816761 32341 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:53.833757 29003 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:53.820214 31629 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:53.848554 29154 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:53.799568 32372 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.820091 31976 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:53.852016 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 45566.1us
I20260709 13:11:53.861629 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.862438 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.802151 32178 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.803072 32343 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:11:53.870028 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.867331 28975 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:53.872859 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.878134 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.873612 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.803435 31529 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.808357 32308 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.876137 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1881) took 1444 ms. Trace:
I20260709 13:11:53.878970 29003 raft_consensus.cc:2376] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8. Re-sending same reply.
I20260709 13:11:53.853802 32098 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:53.887612 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.891711 29001 raft_consensus.cc:2368] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:11:53.880524 28975 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:53.861925 32387 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:53.895643 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.896592 29005 raft_consensus.cc:2368] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8.
I20260709 13:11:53.899897 29152 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:53.889195 29010 rpcz_store.cc:276] 0709 13:11:52.431444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.434966 (+  3522us) service_pool.cc:224] Handling call
0709 13:11:52.435360 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:52.802587 (+367227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:52.802630 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:52.803182 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:11:53.296586 (+493404us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:53.296638 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.296726 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:53.875840 (+579114us) raft_consensus.cc:1685] finished
0709 13:11:53.875868 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:53.876033 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153626,"raft.queue_time_us":176525,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"thread_start_us":706154,"threads_started":2,"wal-append.queue_time_us":687616}
I20260709 13:11:53.901933 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.826731 32023 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:53.902542 29010 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:53.856024 32323 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:11:53.853122 32351 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.904268 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:53.911185 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.913962 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.914595 29003 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:53.889636 32043 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:53.916486 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.917651 29009 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:11:53.919375 29154 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:53.893637 32214 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:53.919719 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.895516 32387 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.907019 32238 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:53.930809 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.934386 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.946599 29009 raft_consensus.cc:2368] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 5. Current term is 6.
I20260709 13:11:53.943789 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.905223 32098 raft_consensus.cc:697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:53.948684 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1731) took 1103 ms. Trace:
I20260709 13:11:53.949203 29078 rpcz_store.cc:276] 0709 13:11:52.844981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.921002 (+ 76021us) service_pool.cc:224] Handling call
0709 13:11:52.921287 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:52.921603 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:53.224360 (+302757us) log.cc:844] Serialized 14 byte log entry
0709 13:11:53.413524 (+189164us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:53.414001 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:11:53.414139 (+   138us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:53.414190 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.414259 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:53.948416 (+534157us) raft_consensus.cc:1685] finished
0709 13:11:53.948444 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:53.948613 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61424,"prepare.queue_time_us":134749,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":23,"raft.queue_time_us":5159,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":2176,"thread_start_us":376457,"threads_started":2,"wal-append.queue_time_us":279169}
I20260709 13:11:53.911078 32368 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.955476 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.955873 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:53.957695 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:53.958949 29159 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:53.920692 32043 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.960474 29156 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:53.960973 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:53.969635 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.970902 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:53.971582 29077 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:53.971527 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:53.972254 29080 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:53.972335 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:53.973197 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:53.965962 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:53.973884 29079 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:53.974903 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:53.975453 29080 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:11:53.977226 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:53.978466 29077 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:11:53.979436 29009 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:53.970968 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.980858 29010 raft_consensus.cc:2368] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8.
I20260709 13:11:53.981839 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.983417 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.985240 29009 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:53.986999 29009 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:53.992939 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:53.998471 29048 leader_election.cc:336] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.717s (SENT)
I20260709 13:11:53.999120 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:54.001672 32377 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.395s	user 0.007s	sys 0.006s
I20260709 13:11:54.002323 32345 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:54.002952 32345 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:54.008355 29080 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:53.960094 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:54.011730 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 495) took 1200 ms. Trace:
I20260709 13:11:54.014377 29081 rpcz_store.cc:276] 0709 13:11:52.810875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.810988 (+   113us) service_pool.cc:224] Handling call
0709 13:11:52.811246 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.436946 (+625700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.011512 (+574566us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b540034d854. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:54.011679 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382397312}
I20260709 13:11:54.013877 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.016333 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.016925 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:54.017441 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.017750 29158 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:54.018498 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.019265 29003 raft_consensus.cc:2410] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:11:54.019555 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:54.020352 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:54.020627 29050 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:53.957297 32206 raft_consensus.cc:697] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:53.943524 32387 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:54.020028 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:54.022596 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 49982.4us
I20260709 13:11:54.016981 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:54.017379 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.023584 29078 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:54.024734 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:54.026935 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.027360 29156 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:54.031232 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:54.031944 29152 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:54.029342 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:54.033233 29156 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:54.019332 32268 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:53.907760 31629 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:53.964226 32043 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:54.049399 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 58350.5us
W20260709 13:11:54.049556 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 61024.1us
I20260709 13:11:54.052304 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.053913 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:54.060917 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:54.105405 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44597.7us
I20260709 13:11:54.032263 32311 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:54.099136 29076 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:11:54.103405 29082 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:11:54.071056 29150 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:54.032377 32317 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:11:54.053277 32359 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:11:54.106701 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 86133us
I20260709 13:11:54.108273 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:53.965344 32368 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:54.021844 32206 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:54.033823 32314 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:11:54.115330 32231 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:54.337478 32231 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:54.116940 32098 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:54.388294 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 47406us
W20260709 13:11:54.113267 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 89823.7us
I20260709 13:11:54.401862 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:54.400887 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:54.182415 29152 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:11:54.240630 29159 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:11:54.444499 29157 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
W20260709 13:11:54.455490 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53744.6us
I20260709 13:11:54.456377 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.060200 29158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:54.472806 29158 raft_consensus.cc:606] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:54.474081 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.474793 29009 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:54.080798 31976 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.333s
I20260709 13:11:54.481570 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.028327 32345 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:54.488094 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 474493us
I20260709 13:11:54.546444 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.470932 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:54.240439 32024 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:54.402971 31910 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:54.553517 32148 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:54.555086 32394 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.315s	user 0.000s	sys 0.000s
W20260709 13:11:54.559614 32391 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.615s	user 0.001s	sys 0.000s
W20260709 13:11:54.560598 32401 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.458s	user 0.000s	sys 0.001s
I20260709 13:11:54.565532 32368 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:54.566143 32231 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:54.464263 32356 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:54.476874 32178 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:54.571766 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.303117 32311 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:54.550714 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 79694us
W20260709 13:11:54.648411 29077 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.068s	sys 0.151s
I20260709 13:11:54.547362 29001 raft_consensus.cc:2376] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9. Re-sending same reply.
I20260709 13:11:54.478353 32392 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:11:54.581382 29085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.018s	sys 0.069s
W20260709 13:11:54.651223 29085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.018s	sys 0.069s
W20260709 13:11:54.584122 29154 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.000s
I20260709 13:11:54.652321 29154 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
W20260709 13:11:54.588780 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 485859us
I20260709 13:11:54.586211 31529 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:54.483666 32398 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.646673 29009 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
W20260709 13:11:54.649724 29077 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.069s	sys 0.151s
I20260709 13:11:54.341400 32268 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.211s
I20260709 13:11:54.659587 32308 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:54.763288 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 306548us
W20260709 13:11:54.737460 32043 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.639s	user 0.006s	sys 0.001s
W20260709 13:11:54.750244 29003 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.000s
I20260709 13:11:54.659369 32351 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:54.791129 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2216) took 1964 ms. Trace:
I20260709 13:11:54.647970 32231 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:11:54.790102 32402 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.485s	user 0.000s	sys 0.001s
I20260709 13:11:54.798136 32317 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:54.799068 29151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.016s	sys 0.014s
W20260709 13:11:54.836687 29151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.016s	sys 0.014s
I20260709 13:11:54.655429 32023 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:54.591434 32148 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:54.795850 32406 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:54.821518 29003 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:54.844528 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.845589 29003 raft_consensus.cc:2393] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:54.846537 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.658421 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.831910 29153 rpcz_store.cc:276] 0709 13:11:52.827029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.309080 (+482051us) service_pool.cc:224] Handling call
0709 13:11:53.388281 (+ 79201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:53.388564 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:53.388604 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:53.389168 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:11:53.874705 (+485537us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:53.874761 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.874846 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:54.790843 (+915997us) raft_consensus.cc:1685] finished
0709 13:11:54.790864 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.791052 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":431061,"threads_started":1,"wal-append.queue_time_us":543881}
I20260709 13:11:54.850162 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.850906 29003 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:54.837522 32360 raft_consensus.cc:2243] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:54.837393 29084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.010s	sys 0.025s
I20260709 13:11:54.852214 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:54.853559 29084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.010s	sys 0.025s
I20260709 13:11:54.854296 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.811691 32368 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:11:54.652143 32356 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:54.801316 32118 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.803821 32311 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:54.833261 32043 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:54.839748 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 398759us
I20260709 13:11:54.797467 32398 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:54.850338 29083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.004s	sys 0.016s
I20260709 13:11:54.858729 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.851492 32268 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:54.843215 32231 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:54.849957 32406 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:54.850067 32209 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:54.844316 32023 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:54.873463 29083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.004s	sys 0.016s
I20260709 13:11:54.842095 32234 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:54.880352 32341 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:54.882506 29682 connection.cc:256] RPC call timeout handler was delayed by -1.077589200000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 13:11:54.885125 32148 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:54.888957 32360 raft_consensus.cc:2272] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:54.892275 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.894614 32398 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:54.897038 32372 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:54.897895 32238 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.944s	user 0.010s	sys 0.000s
I20260709 13:11:54.868119 32415 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:54.899314 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.901327 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 473) took 2046 ms. Trace:
I20260709 13:11:54.902920 32417 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:54.903431 32368 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:54.903954 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1491) took 1125 ms. Trace:
I20260709 13:11:54.904743 32317 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:54.905133 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:54.913219 32209 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:54.983155 29006 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:54.960464 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:54.984784 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:54.976004 29002 rpcz_store.cc:276] 0709 13:11:52.854854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.855575 (+   721us) service_pool.cc:224] Handling call
0709 13:11:53.755854 (+900279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.973485 (+217631us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:53.973535 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:53.973606 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:53.973663 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.973746 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.901133 (+927387us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54003e21d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:54.901246 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17797,"raft.queue_time_us":39498,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4004,"spinlock_wait_cycles":2069696896,"thread_start_us":136668,"threads_started":1}
I20260709 13:11:54.980396 29150 rpcz_store.cc:276] 0709 13:11:53.784441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.784578 (+   137us) service_pool.cc:224] Handling call
0709 13:11:53.806087 (+ 21509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.230630 (+424543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.903733 (+673103us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b5400055d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:54.903900 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607478016}
W20260709 13:11:54.967186 32416 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.055s	user 0.000s	sys 0.003s
W20260709 13:11:54.968490 32422 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.003s
I20260709 13:11:54.869248 32118 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:54.973136 32383 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.247s	user 0.010s	sys 0.010s
I20260709 13:11:54.959316 32404 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.961200 32392 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:54.996371 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:55.001068 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:54.982271 32314 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:54.914546 32268 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:54.973287 32234 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.035388 29076 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:55.036660 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:55.002164 32098 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:54.967748 32023 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.007505 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.003511 32425 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:11:54.981099 32238 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:55.037313 29085 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:54.994622 32341 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:55.048623 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.039199 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:54.983385 32417 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.047200 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:55.053253 29083 raft_consensus.cc:2410] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:11:55.047674 29081 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:55.055605 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:54.985456 32368 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.006384 32206 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:11:55.018858 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 145455us
I20260709 13:11:55.049877 29076 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:55.061420 29078 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: [9.4-9.4]
W20260709 13:11:55.031890 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 197671us
I20260709 13:11:55.015501 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.051047 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:55.062381 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.983505 32415 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.032449 32314 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:55.058210 29081 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:55.063496 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:55.067138 29079 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:55.063095 29005 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:55.064540 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 468) took 2226 ms. Trace:
I20260709 13:11:55.046728 32203 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:55.068148 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:55.068324 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:55.070895 29708 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:55.070600 29712 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
W20260709 13:11:55.070931 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 84627.7us
I20260709 13:11:55.068267 29008 rpcz_store.cc:276] 0709 13:11:52.838549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.840168 (+  1619us) service_pool.cc:224] Handling call
0709 13:11:52.840429 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:52.972511 (+132082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:52.972541 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:52.981127 (+  8586us) log.cc:844] Serialized 26 byte log entry
0709 13:11:53.421907 (+440780us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:53.421946 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.422010 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:54.897983 (+1475973us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:55.056770 (+158787us) raft_consensus.cc:1685] finished
0709 13:11:55.056799 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.064476 (+  7677us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50227,"raft.queue_time_us":92695,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":25819,"spinlock_wait_cycles":113988352,"thread_start_us":428907,"threads_started":2,"wal-append.queue_time_us":529721}
W20260709 13:11:55.063040 32353 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.302s	user 0.011s	sys 0.001s
I20260709 13:11:55.077373 29083 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:55.070362 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:55.074193 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.078341 29081 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:55.079172 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:55.079914 29008 raft_consensus.cc:2376] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7. Re-sending same reply.
I20260709 13:11:55.083336 29009 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:55.085146 29082 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:55.084811 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:55.089248 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.090318 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:55.091701 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.092367 29003 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:55.092640 29005 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:55.087915 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:55.013739 32404 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.051842 32319 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:11:55.100382 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:55.098161 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:55.105150 29005 raft_consensus.cc:2376] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8. Re-sending same reply.
I20260709 13:11:55.088929 32353 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.106597 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.107509 32353 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.107928 29005 raft_consensus.cc:2410] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:11:55.108201 29004 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:55.108781 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2206) took 1090 ms. Trace:
I20260709 13:11:55.109565 32404 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.104180 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2205) took 1085 ms. Trace:
I20260709 13:11:55.115272 29008 rpcz_store.cc:276] 0709 13:11:54.018546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.089028 (+1070482us) service_pool.cc:224] Handling call
0709 13:11:55.104110 (+ 15082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.116427 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.117635 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.118418 29008 raft_consensus.cc:2368] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8.
I20260709 13:11:55.109338 29005 rpcz_store.cc:276] 0709 13:11:54.018812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.106267 (+1087455us) service_pool.cc:224] Handling call
0709 13:11:55.108688 (+  2421us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.119369 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.119556 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.120975 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.122198 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.123005 29005 raft_consensus.cc:2393] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:55.123436 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1908) took 1107 ms. Trace:
I20260709 13:11:55.127506 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:55.133646 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 49240.4us
I20260709 13:11:55.136255 29005 rpcz_store.cc:276] 0709 13:11:54.028529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.121951 (+1093422us) service_pool.cc:224] Handling call
0709 13:11:55.123378 (+  1427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.140201 29083 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:55.141888 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.142881 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.144656 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2207) took 1125 ms. Trace:
I20260709 13:11:55.143823 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:55.145044 29008 rpcz_store.cc:276] 0709 13:11:54.019086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.141655 (+1122569us) service_pool.cc:224] Handling call
0709 13:11:55.144597 (+  2942us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.145387 29082 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 6.
I20260709 13:11:55.043098 32209 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.040380 32406 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.146292 29079 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:55.076306 32203 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.145954 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.055600 32428 consensus_queue.cc:1059] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
W20260709 13:11:55.158034 28976 connection.cc:256] RPC call timeout handler was delayed by -1.564218465999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
W20260709 13:11:55.162379 28976 connection.cc:256] RPC call timeout handler was delayed by -1.5336184659998935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
W20260709 13:11:55.162837 28976 connection.cc:256] RPC call timeout handler was delayed by -1.5111184659999708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 13:11:55.144058 29005 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:11:55.163187 28976 connection.cc:256] RPC call timeout handler was delayed by -1.3959184659998982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 13:11:55.163666 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1904) took 1148 ms. Trace:
I20260709 13:11:55.163777 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:55.164568 29005 rpcz_store.cc:276] 0709 13:11:54.015157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.142549 (+1127392us) service_pool.cc:224] Handling call
0709 13:11:55.163611 (+ 21062us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.165302 28976 leader_election.cc:400] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9.
I20260709 13:11:55.165657 28976 leader_election.cc:403] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:11:55.166700 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:55.168684 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2210) took 1148 ms. Trace:
I20260709 13:11:55.170044 29005 rpcz_store.cc:276] 0709 13:11:54.019831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.166389 (+1146558us) service_pool.cc:224] Handling call
0709 13:11:55.168617 (+  2228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.172669 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:55.176118 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 79989.9us
I20260709 13:11:55.145478 32043 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.177707 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:55.150352 32353 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:55.130745 32319 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:11:55.069226 32392 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.282887 29005 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:55.283874 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.285034 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.285964 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.148834 32314 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.214303 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 514) took 1224 ms. Trace:
W20260709 13:11:55.288985 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 202314us
I20260709 13:11:55.287847 29077 rpcz_store.cc:276] 0709 13:11:53.989987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.000102 (+ 10115us) service_pool.cc:224] Handling call
0709 13:11:54.000491 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:54.801296 (+800805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.801327 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:54.829601 (+ 28274us) log.cc:844] Serialized 26 byte log entry
0709 13:11:54.982355 (+152754us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:54.982397 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.982466 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.214032 (+231566us) raft_consensus.cc:1685] finished
0709 13:11:55.214058 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.214231 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23835,"raft.queue_time_us":337661,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":100,"thread_start_us":911040,"threads_started":2,"wal-append.queue_time_us":297232}
W20260709 13:11:55.288017 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 122861us
I20260709 13:11:55.158691 32209 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:55.294275 29050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.009s	sys 0.013s
W20260709 13:11:55.295301 29050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.009s	sys 0.013s
W20260709 13:11:55.209389 32431 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:11:55.287925 32434 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9.
I20260709 13:11:55.274928 32169 raft_consensus.cc:3055] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:55.215574 32098 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:55.175858 32231 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.216169 32118 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:55.304508 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.200650 32425 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:55.210392 32413 log.cc:927] Time spent T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 13:11:55.210733 32418 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 13:11:55.209920 32432 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:11:55.175565 32436 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.177811 32444 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:55.174602 32238 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
I20260709 13:11:55.325121 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:55.299798 32234 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.324204 32023 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.310065 32169 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:55.312608 32268 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:55.333766 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 156126us
I20260709 13:11:55.333982 29077 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:55.292999 32353 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.323714 32425 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:55.321161 32098 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.343907 29082 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:55.338166 32238 consensus_queue.cc:1243] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:55.337078 32368 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:55.353730 32387 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.308s	user 0.013s	sys 0.000s
I20260709 13:11:55.339897 32231 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:11:55.385867 32387 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.387559 32345 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:55.349112 32319 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:11:55.355399 32203 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.341037 32425 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:55.358182 32417 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.361476 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2222) took 1400 ms. Trace:
I20260709 13:11:55.363641 29002 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:55.329165 32434 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:11:55.398571 29157 rpcz_store.cc:276] 0709 13:11:53.960811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.962226 (+  1415us) service_pool.cc:224] Handling call
0709 13:11:54.444260 (+482034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.465192 (+ 20932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.359993 (+894801us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540041a5d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:55.361395 (+  1402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983318912}
I20260709 13:11:55.376144 32353 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.378763 32169 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.379853 32354 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:55.389137 29006 raft_consensus.cc:2368] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:11:55.378492 32098 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.403744 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2223) took 1442 ms. Trace:
I20260709 13:11:55.346889 32448 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:55.405706 29151 rpcz_store.cc:276] 0709 13:11:53.961085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.962217 (+  1132us) service_pool.cc:224] Handling call
0709 13:11:53.962596 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:53.962952 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:53.962994 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:53.963555 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:11:54.837052 (+873497us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:55.012112 (+175060us) log.cc:844] Serialized 14 byte log entry
0709 13:11:55.036472 (+ 24360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.036701 (+   229us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.403540 (+366839us) raft_consensus.cc:1685] finished
0709 13:11:55.403561 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.403689 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":222904,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":967,"raft.queue_time_us":15640,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":47841,"thread_start_us":946498,"threads_started":2,"wal-append.queue_time_us":955886}
I20260709 13:11:55.408795 32169 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:55.411465 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:55.413630 32440 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:55.414076 32440 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:55.414951 32440 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.416957 32360 ts_tablet_manager.cc:1916] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:11:55.420511 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 362382us
I20260709 13:11:55.421885 32408 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.422405 32408 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.333122 32436 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.424297 32387 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:55.422887 32434 raft_consensus.cc:3055] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:55.443334 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1758) took 1468 ms. Trace:
I20260709 13:11:55.396209 32450 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:11:55.446411 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 110778us
W20260709 13:11:55.453506 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 152876us
I20260709 13:11:55.454802 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:55.466094 29682 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:55.467437 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:55.441082 32203 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:55.442423 32025 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.473349 29085 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:55.454500 29084 rpcz_store.cc:276] 0709 13:11:53.974844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.974968 (+   124us) service_pool.cc:224] Handling call
0709 13:11:54.017798 (+ 42830us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:11:54.853916 (+836118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.853944 (+    28us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:11:54.858575 (+  4631us) log.cc:844] Serialized 104 byte log entry
0709 13:11:54.973863 (+115288us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:55.002003 (+ 28140us) log.cc:844] Serialized 14 byte log entry
0709 13:11:55.008774 (+  6771us) log.cc:844] Serialized 14 byte log entry
0709 13:11:55.366509 (+357735us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b1800450a60. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112754522 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:11:55.415925 (+ 49416us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b1800450a60. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:11:55.441694 (+ 25769us) log.cc:844] Serialized 14 byte log entry
0709 13:11:55.443022 (+  1328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.443098 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.443137 (+    39us) raft_consensus.cc:1685] finished
0709 13:11:55.443149 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.443283 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106322,"prepare.queue_time_us":73156,"prepare.run_cpu_time_us":798,"prepare.run_wall_time_us":3792,"raft.queue_time_us":744739,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":11110,"spinlock_wait_cycles":35216128,"thread_start_us":931985,"threads_started":2,"wal-append.queue_time_us":180011}
I20260709 13:11:55.442903 32434 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:55.444753 32148 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.474642 32185 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:55.480885 32185 raft_consensus.cc:697] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:55.481611 32436 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.482026 32185 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.480841 32406 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.483457 32406 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.485857 32241 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.487003 32406 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.490728 32434 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.407683 32354 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.443431 32440 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:55.470685 32448 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:55.499894 32434 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:55.500406 32355 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:55.501307 32282 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:55.503134 32354 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.503563 32407 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:55.503994 32407 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:55.513880 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 188809us
I20260709 13:11:55.515710 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:55.476768 32025 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.518359 32407 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.519686 32407 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:55.519901 32025 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:55.475584 32203 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.522841 32203 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:55.525020 32382 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 1.921s	user 0.006s	sys 0.024s
I20260709 13:11:55.527278 32356 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:55.534909 32400 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.967s	user 0.000s	sys 0.002s
I20260709 13:11:55.537182 32408 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.537427 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2199) took 1520 ms. Trace:
I20260709 13:11:55.537767 29006 rpcz_store.cc:276] 0709 13:11:54.016956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.996107 (+979151us) service_pool.cc:224] Handling call
0709 13:11:55.537242 (+541135us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b540035e3d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:55.537368 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870308736}
I20260709 13:11:55.538803 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.541359 31629 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:55.545204 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1900) took 1531 ms. Trace:
I20260709 13:11:55.539809 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.546162 29006 raft_consensus.cc:2393] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:11:55.550227 32381 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.821s	user 0.003s	sys 0.008s
I20260709 13:11:55.551210 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.551862 29006 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:55.545590 29005 rpcz_store.cc:276] 0709 13:11:54.014227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.286849 (+1272622us) service_pool.cc:224] Handling call
0709 13:11:55.293066 (+  6217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:55.293350 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.293379 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:55.297005 (+  3626us) log.cc:844] Serialized 26 byte log entry
0709 13:11:55.405386 (+108381us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:55.405424 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.405505 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.545002 (+139497us) raft_consensus.cc:1685] finished
0709 13:11:55.545021 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.545149 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1859,"raft.queue_time_us":28100,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3981,"thread_start_us":85064,"threads_started":1,"wal-append.queue_time_us":158202}
I20260709 13:11:55.552979 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 478) took 2379 ms. Trace:
I20260709 13:11:55.553191 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.553874 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1905) took 1538 ms. Trace:
I20260709 13:11:55.496202 32148 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.554337 29005 rpcz_store.cc:276] 0709 13:11:54.015404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.553170 (+1537766us) service_pool.cc:224] Handling call
0709 13:11:55.553440 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.553590 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.553619 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:55.553662 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:55.553690 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.553738 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.553827 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.553516 29007 rpcz_store.cc:276] 0709 13:11:53.173916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.243095 (+ 69179us) service_pool.cc:224] Handling call
0709 13:11:53.243493 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:53.249585 (+  6092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:53.249629 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:53.250197 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:11:53.486581 (+236384us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:53.486623 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.486697 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.549575 (+2062878us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:55.552625 (+  3050us) raft_consensus.cc:1685] finished
0709 13:11:55.552650 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.552907 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5600,"raft.queue_time_us":10048,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":27268,"spinlock_wait_cycles":54227712,"thread_start_us":236054,"threads_started":1,"wal-append.queue_time_us":326666}
I20260709 13:11:55.500909 32355 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:55.556510 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1907) took 1540 ms. Trace:
I20260709 13:11:55.556571 32148 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:55.556787 32355 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.556983 29007 rpcz_store.cc:276] 0709 13:11:54.015863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.555594 (+1539731us) service_pool.cc:224] Handling call
0709 13:11:55.555828 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.555962 (+   134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.555982 (+    20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:55.556017 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:55.556060 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.556103 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.556460 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.557940 32355 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:55.558774 29007 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:11:55.559398 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2204) took 1541 ms. Trace:
I20260709 13:11:55.560159 31910 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:55.553880 29006 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:55.560927 31910 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.561862 29006 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:55.562647 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1910) took 1518 ms. Trace:
I20260709 13:11:55.550112 32202 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.563371 32202 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.564579 29010 raft_consensus.cc:2410] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:55.559758 29007 rpcz_store.cc:276] 0709 13:11:54.018264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.558126 (+1539862us) service_pool.cc:224] Handling call
0709 13:11:55.558450 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.559180 (+   730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.559351 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.548864 32395 log.cc:927] Time spent T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 1.307s	user 0.002s	sys 0.001s
I20260709 13:11:55.574723 32356 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.575263 32356 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.577286 32356 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.577888 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 487) took 1506 ms. Trace:
I20260709 13:11:55.578038 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1911) took 1509 ms. Trace:
I20260709 13:11:55.578308 29001 rpcz_store.cc:276] 0709 13:11:54.071649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.577129 (+1505480us) service_pool.cc:224] Handling call
0709 13:11:55.577423 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.577583 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.577605 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:55.577642 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:55.577671 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.577733 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.577840 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.580587 32386 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.832s	user 0.000s	sys 0.017s
I20260709 13:11:55.583498 29001 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:55.592896 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2216) took 1010 ms. Trace:
I20260709 13:11:55.593318 29003 rpcz_store.cc:276] 0709 13:11:54.582668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.858437 (+275769us) service_pool.cc:224] Handling call
0709 13:11:55.592740 (+734303us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540032cb54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:55.592836 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503413632}
I20260709 13:11:55.594082 32423 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:55.594558 32423 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:11:55.595824 29009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 1898) took 1582 ms (client timeout 2109 ms). Trace:
W20260709 13:11:55.596704 29009 rpcz_store.cc:269] 0709 13:11:54.013764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.658098 (+644334us) service_pool.cc:224] Handling call
0709 13:11:55.595657 (+937559us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400185bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:55.595765 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488114432}
I20260709 13:11:55.598474 29076 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:55.607487 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1912) took 1500 ms. Trace:
I20260709 13:11:55.608026 29001 rpcz_store.cc:276] 0709 13:11:54.083617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.582888 (+1499271us) service_pool.cc:224] Handling call
0709 13:11:55.583195 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.583850 (+   655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.584021 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.608808 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.517067 28976 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:11:55.567239 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 74254.7us
I20260709 13:11:55.568684 32448 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:55.569134 32454 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:55.851274 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.852239 29076 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
W20260709 13:11:55.717830 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 295426us
I20260709 13:11:55.831337 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2218) took 1205 ms. Trace:
I20260709 13:11:55.877399 29002 rpcz_store.cc:276] 0709 13:11:54.625870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.001759 (+375889us) service_pool.cc:224] Handling call
0709 13:11:55.831149 (+829390us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b540032cdd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:55.831275 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728275328}
W20260709 13:11:55.623335 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 211923us
W20260709 13:11:55.814203 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.878638 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.878582 29122 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:55.879733 29048 leader_election.cc:336] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.873s (SENT)
I20260709 13:11:55.880183 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:55.578954 29007 rpcz_store.cc:276] 0709 13:11:54.068381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.566019 (+1497638us) service_pool.cc:224] Handling call
0709 13:11:55.566347 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.566541 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.566568 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:55.574543 (+  7975us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:55.577833 (+  3290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.577896 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.577992 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.725062 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.626715 29085 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:11:55.666626 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.879765 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:55.873648 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1906) took 1858 ms. Trace:
I20260709 13:11:55.609697 29001 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:11:55.883916 29005 rpcz_store.cc:276] 0709 13:11:54.015640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.555342 (+1539702us) service_pool.cc:224] Handling call
0709 13:11:55.555745 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.555897 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.555919 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:55.555954 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:55.555982 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.556028 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.873576 (+317548us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.885140 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2214) took 1400 ms. Trace:
I20260709 13:11:55.885538 29005 pending_rounds.cc:85] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 6
I20260709 13:11:55.885550 29001 rpcz_store.cc:276] 0709 13:11:54.485206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.608959 (+1123753us) service_pool.cc:224] Handling call
0709 13:11:55.609299 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:55.884909 (+275610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.885092 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.886000 29005 pending_rounds.cc:107] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:11:55.886946 29005 raft_consensus.cc:2889] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:55.890056 29001 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:55.875900 29006 rpcz_store.cc:276] 0709 13:11:54.044482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.561429 (+1516947us) service_pool.cc:224] Handling call
0709 13:11:55.561713 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.562427 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.562587 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.883060 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 366038us
W20260709 13:11:55.884141 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.900731 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2225) took 1101 ms. Trace:
I20260709 13:11:55.901684 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:55.902597 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:55.904131 29085 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:11:55.903589 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:55.902516 29002 rpcz_store.cc:276] 0709 13:11:54.799333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.878139 (+1078806us) service_pool.cc:224] Handling call
0709 13:11:55.900269 (+ 22130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.900437 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.900461 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:55.900499 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:55.900531 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.900590 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.900684 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.907478 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.908082 29048 connection.cc:256] RPC call timeout handler was delayed by -1.2751865540000153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3287s for a response
W20260709 13:11:55.908516 29048 connection.cc:256] RPC call timeout handler was delayed by -1.273916010999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 13:11:55.909034 29048 connection.cc:256] RPC call timeout handler was delayed by -1.2307160110000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:11:55.909499 29048 connection.cc:256] RPC call timeout handler was delayed by -1.1524160110000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 13:11:55.854372 32398 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.006s	sys 0.006s
W20260709 13:11:55.910178 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.911401 29048 connection.cc:256] RPC call timeout handler was delayed by -1.0026457889999847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
I20260709 13:11:55.914862 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:55.914925 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:55.916007 29006 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:11:55.859127 32356 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.916491 29076 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:55.923558 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:55.968485 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 64954.3us
I20260709 13:11:55.969236 29122 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:55.969372 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:55.965231 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:55.971029 29077 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:55.974117 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:55.974687 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:11:55.979153 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55659.2us
I20260709 13:11:55.790237 32427 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:55.667949 32408 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:55.913748 32467 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:55.988261 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:55.991313 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:55.967901 32345 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 13:11:55.900373 32327 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:56.019905 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:55.906038 32268 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:55.914551 32398 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.008s	sys 0.006s
I20260709 13:11:55.952946 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:55.993052 32025 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:55.967449 32419 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:55.972539 32203 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:55.982227 32464 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:56.033608 29085 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:55.891428 32454 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.977463 32356 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:55.975869 31910 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:55.986274 32241 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:56.005286 32462 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.006313 32448 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.968400 32406 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 13:11:56.012655 32384 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 2.286s	user 0.004s	sys 0.012s
I20260709 13:11:55.911185 32118 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.033874 32405 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.667976 32425 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:55.918100 32148 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:55.992952 32467 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:55.989415 32469 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:56.069331 32025 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:56.023406 32327 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.068182 32345 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:56.030975 32317 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:55.984834 32427 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:55.901466 32288 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:56.071810 32464 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.022027 32408 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.095942 32467 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:56.103937 32327 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:56.221447 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:56.326735 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 105863us
W20260709 13:11:56.222718 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:56.328953 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:56.329896 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.330950 32389 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:56.223677 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:56.332212 29001 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
W20260709 13:11:56.332310 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.032512 29002 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:11:56.013803 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2174) took 2234 ms. Trace:
I20260709 13:11:56.225695 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:56.227192 29009 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:56.228353 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 488) took 1172 ms. Trace:
I20260709 13:11:56.039637 29048 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:56.033267 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:56.013497 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.040380 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.010432 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.072952 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.047732 29077 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:11:56.248638 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 258187us
I20260709 13:11:56.098140 32025 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:56.334306 32360 ts_tablet_manager.cc:1929] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:11:56.347563 32360 log.cc:1199] T 2ed915f0be134d71b36d8fc8f7b5830b P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/2ed915f0be134d71b36d8fc8f7b5830b
I20260709 13:11:56.093125 32118 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.094673 32148 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.348587 32148 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:56.350510 32118 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.095476 32405 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.362910 32288 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:56.334823 29010 rpcz_store.cc:276] 0709 13:11:53.779847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.983077 (+203230us) service_pool.cc:224] Handling call
0709 13:11:56.013601 (+2030524us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54001d74d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:56.013745 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3633609984}
I20260709 13:11:56.365111 32288 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.180984 32425 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.150825 32427 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:56.405170 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 77625.7us
I20260709 13:11:56.405704 32427 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:56.185845 32434 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.002s
I20260709 13:11:56.163602 32454 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.093412 32462 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.407598 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:56.096936 32469 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:56.189561 32467 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:56.171691 32472 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:11:56.311264 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1895) took 2298 ms. Trace:
I20260709 13:11:56.413252 29004 rpcz_store.cc:276] 0709 13:11:54.012946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.026787 (+ 13841us) service_pool.cc:224] Handling call
0709 13:11:54.027062 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.108580 (+1081518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.310501 (+1201921us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003aaf54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:56.311028 (+   527us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485799168}
I20260709 13:11:56.313444 32355 consensus_queue.cc:1059] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:56.415354 32355 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:56.415419 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1915) took 2305 ms. Trace:
I20260709 13:11:56.415818 32355 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:56.415843 29003 rpcz_store.cc:276] 0709 13:11:54.109388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.594202 (+1484814us) service_pool.cc:224] Handling call
0709 13:11:55.594550 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:55.595297 (+   747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.595330 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:55.850404 (+255074us) log.cc:844] Serialized 26 byte log entry
0709 13:11:55.972668 (+122264us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:55.972713 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.972791 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:56.414941 (+442150us) raft_consensus.cc:1685] finished
0709 13:11:56.414987 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:56.415224 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25092,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4226,"thread_start_us":121947,"threads_started":1,"wal-append.queue_time_us":222063}
I20260709 13:11:56.282444 32241 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.419291 32241 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.344681 29078 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:56.341820 29080 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:56.313359 32408 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.340969 29076 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:56.342936 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 303222us
I20260709 13:11:56.345350 29085 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
W20260709 13:11:56.167529 32420 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 1.104s	user 0.002s	sys 0.000s
I20260709 13:11:56.364977 32405 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.344027 32356 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.346103 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:56.388159 32389 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.341591 32406 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:56.108889 32464 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:56.425967 32389 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:56.427736 32468 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.912s	user 0.003s	sys 0.026s
I20260709 13:11:56.335943 29007 rpcz_store.cc:276] 0709 13:11:55.055702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.881035 (+825333us) service_pool.cc:224] Handling call
0709 13:11:55.881346 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:55.881684 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.881715 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:55.882165 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:11:56.227696 (+345531us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:56.227824 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.227929 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:56.227991 (+    62us) raft_consensus.cc:1685] finished
0709 13:11:56.228005 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:56.228184 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":15308,"thread_start_us":260022,"threads_started":1,"wal-append.queue_time_us":252668}
I20260709 13:11:56.412544 32469 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:56.425227 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 79449.8us
I20260709 13:11:56.430521 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.430883 32406 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:56.431474 29078 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:56.432624 32406 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:56.434284 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.426175 32241 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.415851 32467 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:56.417387 32355 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.440022 29081 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:56.440057 32323 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:56.440824 32323 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.441087 32454 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:56.441221 32323 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:56.429523 32389 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.446421 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:56.446965 29085 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:56.447937 32354 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:56.448645 32354 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:56.450259 32354 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.456568 29077 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:56.459542 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.460901 29708 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:56.462025 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.465086 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.465895 29712 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:56.467905 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:56.481072 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.474156 32477 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:11:56.482838 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:56.486829 32477 consensus_queue.cc:1243] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:11:56.482266 32314 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:56.509588 32202 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.513098 32282 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:56.515568 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.521696 32203 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:11:56.521725 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65295.8us
I20260709 13:11:56.522327 32203 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:56.523010 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:56.527256 29048 leader_election.cc:400] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8.
I20260709 13:11:56.527591 29048 leader_election.cc:403] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:56.523604 32203 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.528241 32241 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:56.529124 32203 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:56.531847 32282 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:56.526867 32025 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:56.545320 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 138013us
W20260709 13:11:56.550356 32457 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.038s	user 0.000s	sys 0.011s
I20260709 13:11:56.553099 32405 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:56.553182 32241 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8.
I20260709 13:11:56.554764 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1916) took 2149 ms. Trace:
I20260709 13:11:56.559877 32415 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.560915 32355 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:56.555263 29009 rpcz_store.cc:276] 0709 13:11:54.405012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.597444 (+1192432us) service_pool.cc:224] Handling call
0709 13:11:55.634266 (+ 36822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:56.335598 (+701332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.554355 (+218757us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b5400077e54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:11:56.554688 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361744256}
I20260709 13:11:56.568979 32441 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:56.569967 32441 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.575673 32443 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:56.576867 32443 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.570120 32360 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34914 (request call id 73) took 4445 ms. Trace:
I20260709 13:11:56.578401 32462 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.578662 32360 rpcz_store.cc:276] 0709 13:11:52.159815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.271797 (+111982us) service_pool.cc:224] Handling call
0709 13:11:52.976845 (+705048us) ts_tablet_manager.cc:1151] Deleting tablet 2ed915f0be134d71b36d8fc8f7b5830b
0709 13:11:53.038856 (+ 62011us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:11:55.540687 (+2501831us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:56.333425 (+792738us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:56.565438 (+232013us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1301465,"tablet-delete.queue_time_us":229577,"thread_start_us":105583,"threads_started":1}
I20260709 13:11:56.581394 28976 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:56.580999 32210 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:56.569900 32423 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.582470 32415 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:56.583173 32415 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:56.583446 32423 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:56.583632 32415 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:56.584467 32407 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:56.570470 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.586181 32407 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:56.583947 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:56.594229 29076 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:11:56.596069 32364 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:56.596552 32448 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:56.596593 32364 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.598554 32364 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:56.590134 32443 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:56.590340 32441 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.606992 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:56.607167 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80190.3us
I20260709 13:11:56.608151 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.608518 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.609233 29708 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:56.617772 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:56.618165 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:56.630868 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.631832 29085 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:11:56.632632 32282 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:56.633285 32282 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.635825 32406 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:56.636544 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:56.637442 32406 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.639810 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:56.635574 32469 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.669715 29048 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:56.671343 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:56.675426 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.676273 32023 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:56.676326 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.677021 32023 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:56.678274 32023 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.683882 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.684437 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.666795 32406 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:56.712615 29048 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:56.715984 32409 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:56.716578 32412 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:56.720165 32462 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:56.720765 32462 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:56.721230 32462 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:56.723116 32025 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:56.724345 32209 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.725272 32209 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.725793 32467 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:56.729449 32467 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:56.730188 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.730477 32467 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.731916 32467 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:56.732831 32282 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.734519 29708 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:56.735041 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.734555 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:56.737174 32409 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:56.739435 32409 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:56.739811 32409 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:56.740321 32409 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:56.740830 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:56.742328 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 110531us
I20260709 13:11:56.743360 29714 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:11:56.744405 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.746156 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.754530 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.688812 32364 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.687440 32464 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.751663 32209 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.746657 32323 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:56.780560 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 179116us
I20260709 13:11:56.766928 32412 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:56.792486 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:56.792518 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.796427 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:56.787635 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 58614.9us
W20260709 13:11:56.799129 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:56.799660 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.804994 32428 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:11:56.786054 29048 connection.cc:256] RPC call timeout handler was delayed by -1.0481850170000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2721s for a response
I20260709 13:11:56.817032 32364 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:56.818259 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 119437us
W20260709 13:11:56.819662 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.820443 32346 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:56.820828 32346 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:56.821923 32346 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.822922 32346 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:56.823565 32323 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.824180 32415 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.826462 32415 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:56.829579 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.831840 29003 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:56.833178 32412 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.833961 29009 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:56.835619 32412 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.836567 32469 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.837283 32469 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.837844 32469 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.841099 32148 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.843317 32148 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.844772 32209 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.844974 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:56.845448 32209 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.845738 32364 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:56.846589 32364 consensus_queue.cc:1243] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:56.847136 32364 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:56.847208 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:56.847568 32364 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:56.848182 32407 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:56.848649 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:56.848764 32407 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:56.849730 28976 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:56.850010 32407 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.851382 32398 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:56.851938 32398 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:56.852304 32202 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:56.852756 32202 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:56.853138 32202 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:56.852972 32398 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:56.854265 32327 consensus_meta.cc:287] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.525s	user 0.004s	sys 0.010s
I20260709 13:11:56.854669 32327 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.847782 32209 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:56.844823 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.846035 32315 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:11:56.838259 32478 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.405s	user 0.002s	sys 0.000s
I20260709 13:11:56.851487 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:56.859589 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.863022 32354 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:56.863390 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.863659 32354 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.864493 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:56.867339 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61676.4us
I20260709 13:11:56.861604 32315 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:56.868640 32315 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:56.866809 32408 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.874765 32408 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.876456 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:56.877142 29007 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:56.878312 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:56.879060 29007 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:56.856511 32364 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.875792 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:56.839962 32469 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.885842 29004 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:56.865845 32354 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:56.888636 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55943.6us
I20260709 13:11:56.892333 29006 pending_rounds.cc:85] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 3
I20260709 13:11:56.892664 29006 pending_rounds.cc:107] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:11:56.892951 29006 raft_consensus.cc:2889] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:56.877135 32408 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:56.905392 29007 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:56.888958 32323 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:56.908704 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:56.908921 32323 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.910778 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:56.911230 32323 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:56.906008 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:56.912420 29010 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:56.915534 29005 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:56.907815 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:56.919176 32354 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:56.917948 29009 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:56.861191 32462 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.919790 32354 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:56.902195 32282 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.933827 32282 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.934892 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:56.935097 32354 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:56.922192 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:56.919535 28904 catalog_manager.cc:5697] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 2 to 5, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 5 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:56.936251 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:56.936949 29009 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:11:56.934167 29081 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:11:56.938203 29009 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:11:56.938541 29009 pending_rounds.cc:85] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 7
I20260709 13:11:56.899982 32415 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:56.938807 29009 pending_rounds.cc:107] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:11:56.938344 28904 catalog_manager.cc:5697] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 5 to 6, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 6 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:56.950250 29009 raft_consensus.cc:2889] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:56.943173 29157 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:56.938807 32354 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:56.944442 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:56.972774 29008 raft_consensus.cc:2410] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:11:56.975662 29008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2219) took 2325 ms (client timeout 2721 ms). Trace:
I20260709 13:11:56.965437 32354 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:56.963258 32364 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:56.976395 32372 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:56.977696 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:56.977842 29125 connection.cc:256] RPC call timeout handler was delayed by -1.9608196579999913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
I20260709 13:11:56.979194 29153 raft_consensus.cc:2368] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8.
W20260709 13:11:56.979718 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.968170 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:56.941766 32491 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.941493 32323 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.941314 32486 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:56.923791 32315 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:56.985886 29076 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:11:56.968616 32448 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:56.981403 29008 rpcz_store.cc:269] 0709 13:11:54.649786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.145748 (+495962us) service_pool.cc:224] Handling call
0709 13:11:56.975375 (+1829627us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54003f11d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:11:56.975574 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4162225536}
I20260709 13:11:56.984891 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:56.921929 32437 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:11:56.922622 32148 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:11:56.967834 32210 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:56.993733 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 91327.8us
I20260709 13:11:56.922014 32203 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:11:56.986220 29712 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:11:56.978729 32355 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.978936 32415 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:56.996840 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:57.000335 32314 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:56.983220 32354 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.013680 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:56.986245 32327 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.035406 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.001389 29715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 701) took 1763 ms. Trace:
I20260709 13:11:57.024206 29009 raft_consensus.cc:2393] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:57.024216 29007 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.009440 32372 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.001508 32464 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:56.993290 32409 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.000638 32282 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:57.046533 29008 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:57.055096 29008 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:56.993813 32441 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:11:57.032490 32288 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.044479 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.033082 32314 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.052838 32423 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:57.038384 29715 rpcz_store.cc:276] 0709 13:11:55.051128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.051274 (+   146us) service_pool.cc:224] Handling call
0709 13:11:55.051605 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.051890 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:55.053367 (+  1477us) log.cc:844] Serialized 14 byte log entry
0709 13:11:56.469350 (+1415983us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b1800425aa0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:11:56.772942 (+303592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:56.775809 (+  2867us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:56.775854 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.781550 (+  5696us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:56.814751 (+ 33201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1748,"prepare.queue_time_us":1049,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":20696,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"spinlock_wait_cycles":5266176,"thread_start_us":222160,"threads_started":2,"wal-append.queue_time_us":1577117}
W20260709 13:11:57.097052 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 101225us
I20260709 13:11:57.098618 32423 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:57.100487 32423 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:57.101927 32495 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:11:57.046334 32327 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.108642 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.108944 32415 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.109540 32315 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.110167 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.110786 29152 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:57.055902 32372 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.025030 32486 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.112798 29156 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:57.113457 32372 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.113780 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.114050 29157 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:57.114490 32486 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.114897 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.109357 29159 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:57.118026 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.119263 29159 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:57.111790 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.120801 29152 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:57.120746 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:57.121511 29009 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:57.115973 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.124215 32339 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:11:57.125250 32467 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:57.125748 32467 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.126502 32372 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:57.126924 32372 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.127261 32372 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.127748 32467 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:57.140105 29005 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:56.997052 32491 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.033409 32448 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.024758 32323 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.129642 32488 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 13:11:57.130245 29158 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:11:57.133219 29010 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:57.135950 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.142772 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:57.181634 29008 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:11:57.023828 32355 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:57.159621 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 62932.4us
I20260709 13:11:57.098371 32314 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:57.032152 32210 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.131742 32407 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.134570 32148 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:57.191640 32023 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:57.195397 32448 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.196403 29051 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:57.197364 32499 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:11:57.138113 32288 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.200908 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:57.201401 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 68466.2us
I20260709 13:11:57.218271 29005 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:57.201910 32467 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:57.138248 32443 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:11:57.141824 32202 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.108687 32327 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.229628 32202 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.229890 32327 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.211841 32148 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:57.230965 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 135411us
I20260709 13:11:57.231168 32148 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.215893 32407 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.232784 29001 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:57.233897 32407 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.235220 32423 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.235534 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:57.236085 32315 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:57.236476 32315 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:57.236917 32409 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.237535 32412 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:57.237318 32315 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.238071 32412 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:57.238509 32315 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:57.239019 32412 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.240193 32464 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:57.241101 32323 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.241869 32323 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:57.242051 29009 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:57.242342 32323 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:57.241115 32148 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:57.242911 32323 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:57.243665 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:11:57.244308 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.246928 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.247638 29154 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
W20260709 13:11:57.209017 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.184933 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 85707.5us
I20260709 13:11:57.248821 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.207206 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.252684 32404 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:57.253259 32404 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.222036 29002 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:57.226099 32467 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.254990 28904 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 3 to 5, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 5 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:57.255861 32467 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.257632 32462 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.252429 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 2ed915f0be134d71b36d8fc8f7b5830b (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
W20260709 13:11:57.258370 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.227921 32210 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:57.258873 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 278518us
I20260709 13:11:57.259562 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:57.212124 29007 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:11:57.261935 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.267822 29153 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:11:57.275122 29159 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:11:57.281270 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.281741 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.256976 28904 catalog_manager.cc:5697] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 5, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 5 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:57.285537 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.287653 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.288414 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.289108 32491 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.289425 32288 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.289673 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.291406 32404 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.292309 29084 raft_consensus.cc:2410] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:57.294613 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:57.301549 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.302125 29084 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:57.305235 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:57.305455 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:57.305992 32491 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:57.306772 29084 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:57.307029 32491 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:57.307548 32288 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.308125 32023 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.308036 32288 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.308738 32023 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.310063 32288 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:57.310562 32288 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:57.311621 29082 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.305238 32477 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:57.312443 32477 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:57.313469 32477 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.315001 32477 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:57.314770 32314 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.316200 32398 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:57.316397 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.317090 29157 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:11:57.318005 32203 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:57.318579 32203 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:57.310891 32023 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.320572 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:57.324576 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.310882 32288 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:11:57.326262 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.336735 32203 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.337455 32423 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:57.333235 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.349434 29010 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:11:57.349565 29009 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:57.350756 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:57.353900 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.345206 29076 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.355309 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.356359 29004 raft_consensus.cc:2410] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:11:57.358496 29712 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:57.359565 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:57.332497 32025 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:57.360383 32025 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:57.360705 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:57.361213 32025 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.361719 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:57.359953 32504 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:57.358361 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:57.283209 32437 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
W20260709 13:11:57.414093 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.410677 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:57.456331 29002 raft_consensus.cc:2410] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:11:57.468483 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 176884us
I20260709 13:11:57.373561 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:57.463274 29010 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:11:57.639392 32409 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:57.639806 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.640483 29010 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:57.641394 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.641739 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.642023 29010 raft_consensus.cc:2393] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:57.362162 29082 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:57.642946 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:57.357869 29125 connection.cc:256] RPC call timeout handler was delayed by -1.2958184510000592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
W20260709 13:11:57.639472 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 282436us
I20260709 13:11:57.476516 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.362486 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:57.379640 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:57.645928 29709 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.380195 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:57.646625 29713 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
W20260709 13:11:57.385882 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.378209 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.368675 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.532510 29007 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:11:57.535400 29009 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:57.382120 29077 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:57.480116 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.649710 29008 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:57.535203 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.648404 29083 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:57.547199 29076 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:57.369473 32314 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:57.644848 32241 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.656500 29084 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:11:57.656992 32282 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:57.657368 32282 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:57.656566 32241 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.658219 32282 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.659514 32282 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:57.639920 32409 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.661005 32241 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.661224 32409 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:57.662319 32515 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.371s	user 0.000s	sys 0.004s
I20260709 13:11:57.672705 29077 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:57.674226 32423 consensus_queue.cc:1059] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:57.675220 32423 raft_consensus.cc:3055] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:57.675576 32423 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:57.676443 32423 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.552517 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.550956 29122 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:57.377369 32346 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.174s
I20260709 13:11:57.375927 32428 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.179s
I20260709 13:11:57.562330 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.683800 29709 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:57.686589 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.410178 32404 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.149s
I20260709 13:11:57.500619 32288 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:57.569872 32354 consensus_meta.cc:287] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.000s
I20260709 13:11:57.569569 32467 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.446201 32323 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.574205 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.576109 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:57.614638 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.416366 32504 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.587419 32372 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.631071 29158 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:57.637676 32355 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:57.373306 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.726997 29157 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:11:57.644443 29125 connection.cc:256] RPC call timeout handler was delayed by -1.221019657999932s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28500000000000003s for a response
I20260709 13:11:57.655476 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.727584 29156 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:57.727736 29125 connection.cc:256] RPC call timeout handler was delayed by -1.2088318820000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
I20260709 13:11:57.648769 29007 pending_rounds.cc:85] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 4
I20260709 13:11:57.728823 29007 pending_rounds.cc:107] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:11:57.729163 29007 raft_consensus.cc:2889] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:57.691282 29002 raft_consensus.cc:2410] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:11:57.693733 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.694972 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.688961 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 138073us
W20260709 13:11:57.708499 29085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.001s	sys 0.016s
W20260709 13:11:57.731097 29085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.001s	sys 0.016s
W20260709 13:11:57.728142 29125 connection.cc:256] RPC call timeout handler was delayed by -1.1221184510000057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
I20260709 13:11:57.731668 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:57.731999 29125 connection.cc:256] RPC call timeout handler was delayed by -1.0967114350000884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
I20260709 13:11:57.731391 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.733115 29154 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.731379 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.707372 32491 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:57.728092 29150 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:57.738334 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:57.734485 29159 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:11:57.743052 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.743757 29006 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:57.748246 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.706733 32354 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.732360 32519 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.748942 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:57.752225 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.769870 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.773787 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.772327 29157 raft_consensus.cc:2410] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:11:57.774362 29006 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:57.773095 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.776748 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.777225 29002 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:57.772989 29153 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:57.781855 29150 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:57.783412 29002 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:57.779479 29003 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:11:57.751827 32491 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:57.742816 32477 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:11:57.794977 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:57.779213 32288 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.771559 32522 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:57.837966 32522 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:57.838956 32522 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.799930 32354 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:57.801209 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.803165 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.803830 32339 consensus_queue.cc:1059] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:57.840332 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.842193 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.773491 32519 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.815060 32412 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:57.819542 32148 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:57.820523 32323 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:57.821547 32025 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:57.786916 32315 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.829820 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.836180 32467 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:57.837379 32504 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:57.852794 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.856746 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.857290 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.779874 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.865160 29709 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:11:57.889950 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.857510 29683 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:57.870684 29154 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:57.865386 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.891031 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.878625 29708 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
W20260709 13:11:57.839998 32514 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
W20260709 13:11:57.837698 32511 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.284s	user 0.002s	sys 0.000s
I20260709 13:11:57.881163 29153 raft_consensus.cc:2410] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:57.884193 32315 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.885070 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:57.895995 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:57.895785 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 160784us
I20260709 13:11:57.911360 29084 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:57.912485 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:57.913403 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:57.913470 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:57.914319 32364 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:57.914681 32364 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.915024 32364 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.916589 32519 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:57.918440 32488 raft_consensus.cc:3055] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:57.918841 32488 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:57.893587 32148 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:57.921509 32148 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.896476 32491 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.924011 32314 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:57.900785 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.925990 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.904313 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.926914 29155 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:57.927708 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.928819 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.929816 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.930322 29155 raft_consensus.cc:2376] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8. Re-sending same reply.
I20260709 13:11:57.898351 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.905789 28904 catalog_manager.cc:5697] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 4, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 4 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:57.898643 32372 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:57.908135 32404 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:57.941428 29158 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:57.907884 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:57.908910 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:57.909425 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.901216 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.942907 29080 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:11:57.943097 29150 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:57.943588 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.901503 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.932426 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:57.902164 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.901788 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.946025 29008 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:57.903026 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.897358 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.947923 29009 raft_consensus.cc:2410] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:11:57.898834 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:57.891577 29156 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
W20260709 13:11:57.937052 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 79595.5us
I20260709 13:11:57.866149 32412 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:57.938436 29085 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:11:57.952482 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44696.4us
W20260709 13:11:57.923907 28905 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52674 (request call id 259) took 34200 ms (client timeout 15000 ms). Trace:
W20260709 13:11:57.952992 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.942524 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.944608 29003 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:57.946760 29154 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:57.902426 29006 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: [6.3-8.4]
I20260709 13:11:57.951297 32516 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:57.951638 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:57.953795 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.955309 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.956198 29159 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:57.956087 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.960191 29158 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:11:57.957020 32516 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.961078 32516 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.965279 28904 catalog_manager.cc:5697] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:57.958958 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.967782 29159 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:57.968768 29155 raft_consensus.cc:2410] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:11:57.938923 32423 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.970741 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.877804 32409 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.919610 32488 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.972759 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:57.923413 32364 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:57.969797 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:58.056533 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 144125us
I20260709 13:11:58.000350 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.060973 29156 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:58.046218 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:57.974537 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:58.062985 29155 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.061482 32516 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:57.974073 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.064810 32364 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:58.051267 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 98037.7us
I20260709 13:11:58.074070 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:58.074326 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.065277 32516 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:58.021070 28975 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.767s (SENT) [suppressed 38 similar messages]
I20260709 13:11:57.955369 32404 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:57.957945 32519 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:58.067981 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:58.070241 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:58.078842 29008 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:58.079252 32462 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:58.079396 32408 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.080255 29007 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:11:58.080804 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:11:58.098652 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 102230us
I20260709 13:11:58.098899 29009 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.081429 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:58.084257 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:58.101437 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:57.971233 32346 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.094010 29122 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.062074 32488 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:58.023115 32437 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:58.097375 29077 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:11:58.106851 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 155277us
I20260709 13:11:58.104533 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.108106 29154 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:58.120245 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:58.102569 32372 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:58.095434 32519 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.104264 32241 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:58.104970 32535 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.141469 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:58.142349 29008 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:11:58.080678 28904 catalog_manager.cc:5697] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 2 to 4, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65), VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66), VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67), VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 3 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:58.147269 29154 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 7. Ops: []
W20260709 13:11:58.146885 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53732.1us
I20260709 13:11:58.116660 32364 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:58.119067 32491 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:58.121618 32404 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:58.068440 32448 consensus_queue.cc:1175] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "befda4a6c8344c26b0f139e4e1686b3b" responder_term: 8 status { last_received { term: 8 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 8 index: 8 } }
I20260709 13:11:58.142033 32519 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.173583 29158 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:58.159246 29009 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:58.104203 32462 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.185499 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:58.157922 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:11:58.188320 29008 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.188947 29153 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:11:58.165443 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.161126 32491 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.193429 29158 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:11:58.167541 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 59711.5us
I20260709 13:11:58.146914 32241 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.142867 32372 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:58.171604 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 70166.2us
W20260709 13:11:57.953266 28905 rpcz_store.cc:269] 0709 13:11:23.723910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.728159 (+  4249us) service_pool.cc:224] Handling call
0709 13:11:57.923670 (+34195511us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:52.733922 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:52.734775 (+   853us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:52.734837 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:11:52.734862 (+    25us) tablet.cc:662] Decoding operations
0709 13:11:52.815097 (+ 80235us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:52.829625 (+ 14528us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:52.837217 (+  7592us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:11:52.925053 (+ 87836us) tablet.cc:701] Row locks acquired
0709 13:11:52.925105 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:11:52.925289 (+   184us) write_op.cc:270] Start()
0709 13:11:52.925440 (+   151us) write_op.cc:276] Timestamp: P: 1783602712925223 usec, L: 0
0709 13:11:52.925460 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:11:52.926157 (+   697us) log.cc:844] Serialized 4340 byte log entry
0709 13:11:53.981970 (+1055813us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:11:54.778538 (+796568us) op_driver.cc:464] REPLICATION: finished
0709 13:11:55.200301 (+421763us) write_op.cc:301] APPLY: starting
0709 13:11:55.214435 (+ 14134us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:55.214752 (+   317us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:55.214799 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:55.272649 (+ 57850us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:55.282353 (+  9704us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:11:55.282425 (+    72us) write_op.cc:312] APPLY: finished
0709 13:11:55.282911 (+   486us) log.cc:844] Serialized 77 byte log entry
0709 13:11:55.294484 (+ 11573us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:55.309839 (+ 15355us) write_op.cc:454] Released schema lock
0709 13:11:55.319862 (+ 10023us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:55.341655 (+ 21793us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:55.346962 (+  5307us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2805878,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":353344,"apply.run_cpu_time_us":4686,"apply.run_wall_time_us":155929,"num_ops":10,"prepare.queue_time_us":975944,"prepare.run_cpu_time_us":75993,"prepare.run_wall_time_us":559095,"raft.queue_time_us":645816,"raft.run_cpu_time_us":20660,"raft.run_wall_time_us":399861,"replication_time_us":1766182,"spinlock_wait_cycles":119909120,"thread_start_us":1376989,"threads_started":4,"wal-append.queue_time_us":574539,"wal-append.run_cpu_time_us":71145,"wal-append.run_wall_time_us":2808915}]]}
I20260709 13:11:58.229382 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:58.121995 32346 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:58.180984 32533 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:11:58.181917 28904 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 3 to 5, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 5 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:58.184813 32544 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:58.122848 32408 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.245242 32346 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:58.244772 28904 catalog_manager.cc:5697] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:11:58.245877 32408 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:58.246933 32346 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:58.189874 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.246884 28904 catalog_manager.cc:5697] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:11:58.175433 32355 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:58.196128 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:58.197536 32491 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:58.203135 29154 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:58.206578 32315 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:58.212813 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.213817 29156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:11:58.276744 29156 raft_consensus.cc:606] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:58.105631 32412 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:58.216084 32241 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:58.217290 29008 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:58.217972 32462 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:58.220881 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.117426 32437 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:58.281567 32437 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:58.283144 32346 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:58.283638 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:58.284071 32346 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:58.284484 32346 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:11:58.285348 32346 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.285858 32210 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:58.234061 29157 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:11:58.243850 32544 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.196234 29155 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
W20260709 13:11:58.248431 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63098.7us
I20260709 13:11:58.248550 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.158604 32535 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.252187 32504 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:58.259027 29152 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:11:58.268062 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:58.273669 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 77641.2us
I20260709 13:11:58.274473 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:58.296782 29150 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:58.296653 28976 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:58.298795 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.299227 29159 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:58.300494 29003 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:11:58.240159 32372 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:58.304956 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.306177 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:58.306535 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:58.307125 29155 raft_consensus.cc:2376] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7. Re-sending same reply.
I20260709 13:11:58.307242 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.307945 29157 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:11:58.308923 29157 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 6. Current term is 8. Ops: []
I20260709 13:11:58.287958 29153 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:11:58.298290 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.288820 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:58.310956 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.311386 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:11:58.311939 29154 raft_consensus.cc:2376] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10. Re-sending same reply.
W20260709 13:11:58.293624 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.301713 32441 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:58.308791 28975 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:58.295907 32544 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:58.303259 32535 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:58.318902 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.303907 29125 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:58.304360 32407 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:58.286648 32346 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:58.326997 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.334064 29150 raft_consensus.cc:2410] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:58.311211 29155 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
W20260709 13:11:58.334913 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 105610us
I20260709 13:11:58.313517 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.274178 32552 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:58.336510 29048 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:11:58.314393 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.317684 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:58.273005 32355 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:58.303802 32504 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:58.324875 29005 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:58.325246 29156 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:11:58.323855 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.340129 29156 pending_rounds.cc:85] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 4
I20260709 13:11:58.340427 29156 pending_rounds.cc:107] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:11:58.340764 29156 raft_consensus.cc:2889] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.338102 29151 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:11:58.337669 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:58.317368 32441 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:58.364486 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:58.375659 29004 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:58.324498 32412 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:58.372985 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 49208.5us
I20260709 13:11:58.323922 32148 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:11:58.345752 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 57032.2us
I20260709 13:11:58.378180 29152 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:58.364081 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:58.385411 29708 raft_consensus.cc:2410] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:11:58.347985 29153 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:58.315429 32428 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:11:58.391006 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.343071 29154 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:58.314960 32522 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:11:58.314862 32467 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:11:58.406515 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.408196 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.409147 29156 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:58.411156 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.411983 29153 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:58.344100 32355 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.338356 32025 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:58.405550 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.338049 32323 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:11:58.414850 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 69032.2us
I20260709 13:11:58.417958 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.379530 32535 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:58.379769 32441 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:58.345356 32364 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:11:58.419713 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 102083us
I20260709 13:11:58.336246 32409 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:58.336567 32552 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:11:58.385597 32288 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:58.460179 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:58.461477 29079 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.428382 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.448386 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:58.462711 29152 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:58.463979 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:11:58.430239 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.421391 29050 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:58.421723 29150 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:11:58.337965 32550 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:11:58.324097 32545 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:11:58.386193 32411 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:58.405148 32464 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:58.466622 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 162791us
I20260709 13:11:58.467299 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.419909 32314 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:58.481695 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 96451us
W20260709 13:11:58.481631 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.483788 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:58.427369 32535 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:58.460865 32557 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:58.496541 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:58.490449 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.498860 29158 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:11:58.499958 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.500612 29158 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:11:58.502609 29158 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:11:58.502899 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:58.473867 32315 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:58.468622 32552 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:58.507769 29159 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:11:58.473162 32411 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:58.512441 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:11:58.546622 29150 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:58.481248 32428 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:58.538309 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.548197 32428 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:58.505857 32557 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.550571 32557 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.481547 32288 raft_consensus.cc:697] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:58.468099 32423 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:11:58.491421 32407 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:58.491447 32535 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.475136 32563 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:11:58.552599 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 147524us
I20260709 13:11:58.474910 32464 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:58.582988 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 76524.9us
I20260709 13:11:58.586241 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:58.540270 32315 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.584761 29003 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:58.554646 32411 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.502216 32282 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:11:58.593417 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:58.552395 32323 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:58.557943 32314 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:58.616318 29005 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
W20260709 13:11:58.617709 29006 consensus_meta.cc:287] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.001s
I20260709 13:11:58.618687 29006 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:11:58.481669 32504 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.578526 32559 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:58.552747 32448 consensus_queue.cc:1059] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:11:58.581490 32563 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.620410 29156 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:11:58.603688 32315 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:58.624857 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.586642 32566 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:58.615981 32545 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:58.506623 32409 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:11:58.645295 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:58.628607 32559 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.624079 32560 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:58.585078 32364 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.630734 32563 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:58.631796 32556 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.636746 32148 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:58.487690 28904 catalog_manager.cc:5697] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 4, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 4 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:58.674708 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 178212us
I20260709 13:11:58.676288 28976 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:11:58.678884 32560 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:58.679579 32560 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:58.680768 32560 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:58.674530 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 81533.5us
I20260709 13:11:58.649471 32566 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:58.688870 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:58.672109 32467 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:58.653318 32559 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:58.692457 29682 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:58.697546 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52326.2us
W20260709 13:11:58.686679 32554 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
W20260709 13:11:58.698480 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.701258 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.703446 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:11:58.632998 32565 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:11:58.704594 29682 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:58.690950 32566 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.696152 32467 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.652390 32288 consensus_queue.cc:237] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.702994 32423 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:58.737290 28975 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:11:58.719439 32551 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 13:11:58.652195 32407 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.719362 32571 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.743763 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:11:58.677419 32556 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.733084 32467 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:58.731626 32545 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.690198 32323 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.719792 32566 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:58.737462 32314 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:11:58.750252 32546 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:58.749893 32572 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:58.754483 29079 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:11:58.755194 32575 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:11:58.768406 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.775799 32561 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:58.776278 32561 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.776145 32570 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:11:58.776734 32561 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:58.745081 32443 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 13:11:58.813165 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 327212us
I20260709 13:11:58.728863 32562 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.150s
W20260709 13:11:58.820905 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 457096us
W20260709 13:11:58.822108 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 400721us
W20260709 13:11:58.775792 32553 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:11:58.778376 32464 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:11:58.828616 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.839557 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:58.747515 32148 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.764187 32241 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:58.849269 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 276169us
I20260709 13:11:58.752985 32571 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:58.777837 32567 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.142s
I20260709 13:11:58.752285 32545 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:58.852829 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.829998 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:58.831916 32550 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:58.838860 32564 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:58.855755 29683 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:58.851099 32519 consensus_meta.cc:287] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.004s
I20260709 13:11:58.826555 32546 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:11:58.878899 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:11:58.883127 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53388.9us
I20260709 13:11:58.867923 32464 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:58.860630 32550 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:11:58.939517 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.874300 32575 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.875084 32025 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:58.884769 32372 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.003s
I20260709 13:11:58.881726 32428 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:58.872085 32519 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:59.051611 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 197776us
I20260709 13:11:58.868963 32314 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:58.962461 32564 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:11:59.074591 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:58.974397 32364 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:59.072135 32525 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:59.087502 32525 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:11:59.088254 32589 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:11:59.088423 32525 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:59.071832 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 176283us
W20260709 13:11:59.053421 32558 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:11:59.080700 29157 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:58.980158 32578 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:59.056279 32025 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:59.087435 32241 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:59.104562 32591 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:11:59.105333 32241 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:59.105086 32591 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:59.105731 32241 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:59.106114 32241 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:59.107241 32591 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:59.107940 32579 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.104s	user 0.000s	sys 0.009s
I20260709 13:11:59.103596 32411 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.113672 32564 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.114833 32411 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:59.116221 32354 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:59.119740 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:59.124167 32504 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:59.125499 32504 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:59.130404 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:59.131899 32241 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.134017 32241 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:59.134605 32241 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:11:59.135200 32346 consensus_meta.cc:287] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.789s	user 0.007s	sys 0.000s
I20260709 13:11:59.137200 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:59.137936 29708 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:11:59.139652 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:59.141687 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:59.143134 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:59.143568 32404 consensus_meta.cc:287] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.978s	user 0.008s	sys 0.000s
I20260709 13:11:59.143683 29709 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:11:59.147023 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:58.899638 28904 catalog_manager.cc:5697] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 1 to 3, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 3 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:59.150429 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:59.158246 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:11:59.167310 29085 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:59.176882 32571 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:59.182835 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.059734 32372 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.158861 32578 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:59.188793 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 431211us
I20260709 13:11:59.189298 29085 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:59.189898 32354 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:59.190343 32488 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:59.190590 32354 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.191054 32488 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.193444 32488 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:59.175792 32544 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.833s	user 0.006s	sys 0.005s
I20260709 13:11:58.972960 32575 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:59.199961 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:11:59.186299 32557 consensus_meta.cc:287] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.000s
I20260709 13:11:59.165658 32556 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:59.101505 32589 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:11:59.202517 29085 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:11:59.283329 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 453380us
I20260709 13:11:59.204901 28975 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:59.100716 32565 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:11:59.224082 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:11:59.226750 29125 connection.cc:256] RPC call timeout handler was delayed by -1.2239443409999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 13:11:59.204726 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:11:59.174028 32404 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:59.285194 29125 connection.cc:256] RPC call timeout handler was delayed by -1.187944340999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 13:11:59.159554 32364 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:59.282604 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 208059us
W20260709 13:11:59.202064 32581 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.060s	user 0.000s	sys 0.002s
I20260709 13:11:59.196604 32578 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:59.327739 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:59.332859 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 45374.4us
I20260709 13:11:59.351851 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:59.205143 32355 consensus_meta.cc:287] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.002s
I20260709 13:11:59.347954 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:59.356273 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:11:59.462792 29712 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:11:59.366782 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.214890 32339 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:59.577528 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 251416us
I20260709 13:11:59.578545 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:11:59.581862 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 230095us
W20260709 13:11:59.509570 32441 consensus_meta.cc:287] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.039s	user 0.004s	sys 0.006s
I20260709 13:11:59.099346 32550 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:59.599301 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 271629us
I20260709 13:11:59.600178 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:59.599922 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 271677us
I20260709 13:11:59.543738 32556 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:59.203588 32544 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:59.213821 32557 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:59.580618 32354 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:59.640452 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:59.546226 32464 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:59.353281 32488 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.247s
I20260709 13:11:59.576385 32408 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:11:59.663214 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 57496.5us
W20260709 13:11:59.622645 32148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.020s	sys 0.039s
I20260709 13:11:59.684006 32598 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.327s
I20260709 13:11:59.685671 32603 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:11:59.686254 32603 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.688565 32372 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:59.639144 32571 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:11:59.689150 32593 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:11:59.689731 32571 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:59.690256 32571 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:59.663573 32563 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.965s	user 0.011s	sys 0.000s
I20260709 13:11:59.692375 32563 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:59.694619 32563 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:59.697959 32412 consensus_meta.cc:287] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.296s	user 0.008s	sys 0.003s
W20260709 13:11:59.699172 32491 consensus_meta.cc:287] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.410s	user 0.011s	sys 0.000s
I20260709 13:11:59.699661 32491 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:59.702008 32491 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:59.703200 29083 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:11:59.687559 32315 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.052s	user 0.005s	sys 0.002s
I20260709 13:11:59.537464 32355 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.581432 32339 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:11:59.678258 32535 consensus_meta.cc:287] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.065s	user 0.007s	sys 0.004s
I20260709 13:11:59.752298 32535 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:59.755247 32535 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:59.757251 32339 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:59.649093 32545 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:11:59.760497 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 120120us
I20260709 13:11:59.676844 32464 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:59.769045 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 89290.3us
I20260709 13:11:59.770382 29050 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:11:59.743127 32323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.128s	sys 0.189s
I20260709 13:11:59.682008 32408 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:11:59.785770 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 184080us
W20260709 13:11:59.762742 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 557866us
W20260709 13:11:59.763239 29080 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.000s	sys 0.009s
I20260709 13:11:59.764276 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:11:59.761408 29077 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:11:59.681856 32148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.020s	sys 0.039s
W20260709 13:11:59.791911 29080 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.000s	sys 0.010s
W20260709 13:11:59.796952 32212 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.021s	sys 0.035s
W20260709 13:11:59.797333 32212 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.021s	sys 0.035s
I20260709 13:11:59.797725 32599 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.798293 32599 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:11:59.802909 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:11:59.784857 32323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.128s	sys 0.189s
I20260709 13:11:59.606401 32441 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.761687 32519 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:11:59.769438 32589 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:59.836633 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 66328.7us
I20260709 13:11:59.851046 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:59.844799 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:11:59.852272 29082 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:11:59.789582 32408 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:59.761200 32315 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:11:59.835537 32566 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.063s	user 0.010s	sys 0.000s
I20260709 13:11:59.803565 32412 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:59.893523 28975 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:59.783286 32563 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:11:59.792222 32598 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:11:59.842396 32571 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:59.865708 32404 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.917373 32594 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:59.902393 32412 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:59.925060 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.909061 32407 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:59.919245 32572 consensus_meta.cc:287] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.105s	user 0.010s	sys 0.000s
I20260709 13:11:59.954775 28975 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:11:59.916013 32364 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:11:59.958590 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 107641us
I20260709 13:11:59.959342 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:11:59.900564 32603 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:11:59.923748 32575 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:59.902791 32566 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.917604 32599 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:59.935176 32594 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:11:59.968115 28975 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:11:59.969774 29050 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:11:59.934356 32412 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:59.948745 32586 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:59.948799 32597 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:59.933759 32409 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.915928 32404 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.983786 32575 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:59.986888 32602 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:11:59.987403 32604 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.970880 32407 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:00.011931 32407 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:00.013430 32561 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.236s	user 0.012s	sys 0.000s
I20260709 13:11:59.992592 32599 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:59.992676 32519 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.013921 32561 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.014312 32519 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.002218 32355 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.016690 32519 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.018208 32404 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:00.019778 32467 consensus_meta.cc:287] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.247s	user 0.010s	sys 0.000s
I20260709 13:12:00.012909 32407 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:11:59.993618 32544 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:59.972177 32572 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:59.997208 32557 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:59.982486 32578 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.034238 28975 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:00.034236 29156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2261) took 1631 ms (client timeout 1851 ms). Trace:
W20260709 13:12:00.028358 32025 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.858s	user 0.007s	sys 0.003s
W20260709 13:12:00.036425 29156 rpcz_store.cc:269] 0709 13:11:58.399401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.467096 (+ 67695us) service_pool.cc:224] Handling call
0709 13:12:00.030272 (+1563176us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001cfcd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:00.030377 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347410688}
I20260709 13:12:00.026268 32614 raft_consensus.cc:3055] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:00.026445 32623 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:00.046340 32606 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:12:00.046586 32623 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:00.047946 32621 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:12:00.048545 32605 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:12:00.048950 32623 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.028865 32561 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.043422 32609 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:00.043609 32625 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:00.044296 32627 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:00.045296 32578 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.004961 32409 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.054950 29709 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:00.059036 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 480529us
I20260709 13:12:00.050252 32025 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.061834 32626 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:00.064379 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:00.035956 32315 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:11:59.997586 32594 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.027477 32467 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.050024 32614 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:00.084340 29680 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:00.025076 32604 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.092193 32609 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:00.093415 32604 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.093629 32624 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:00.094056 32624 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:12:00.094323 32559 consensus_meta.cc:287] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.389s	user 0.013s	sys 0.000s
I20260709 13:12:00.094424 32624 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:00.087570 32626 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:00.096163 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica befda4a6c8344c26b0f139e4e1686b3b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:00.095995 32614 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.071455 32625 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:00.096928 28895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 47004.4us
I20260709 13:12:00.097093 32614 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:00.092495 32409 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.094740 32559 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.101389 32563 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:00.101774 32563 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:00.102391 32559 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.102589 32563 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.080103 32623 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:00.088479 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:00.083630 32572 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.103719 32626 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.103771 32563 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.106956 32623 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.086472 32355 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:12:00.122797 32623 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:00.086162 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 125743us
I20260709 13:12:00.097764 32625 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.133342 29050 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:00.134938 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:00.086586 32603 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 13:12:00.136780 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 48385.5us
I20260709 13:12:00.107445 32411 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:00.083779 32611 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:12:00.108630 32630 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:00.133785 32571 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:00.145318 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:00.151315 32025 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:00.142129 32589 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:00.157214 32412 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.159873 32604 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:00.161818 29683 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:00.161964 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.163398 29078 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:00.165601 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1519) took 2432 ms. Trace:
I20260709 13:12:00.166093 29157 rpcz_store.cc:276] 0709 13:11:57.732732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.309615 (+576883us) service_pool.cc:224] Handling call
0709 13:11:58.338811 (+ 29196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.094820 (+756009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.165278 (+1070458us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b54003f70d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:00.165502 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705723392}
I20260709 13:12:00.107092 32572 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.169710 32572 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:00.175488 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.140244 32516 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:00.188649 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.191882 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.192600 29076 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:00.136248 32623 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:00.143744 32566 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:00.211448 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 66190.9us
I20260709 13:12:00.185607 32625 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.212726 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.190045 32590 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:00.215015 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.190536 32282 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:00.218173 32575 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:00.219929 32594 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.188591 32604 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.220687 32467 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:00.185065 32628 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:00.186946 32603 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.170672 32626 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:00.189392 32571 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.233414 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.228615 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:00.213107 32623 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:00.188995 32516 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:00.225375 32590 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:00.235731 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.236850 29078 raft_consensus.cc:2393] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:00.231433 32604 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.237833 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:00.231540 32423 consensus_meta.cc:287] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.490s	user 0.012s	sys 0.000s
I20260709 13:12:00.227555 32412 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.242437 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:00.231271 32628 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.238533 32282 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:00.246953 32516 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.248201 32516 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:00.252450 32025 raft_consensus.cc:993] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:12:00.252951 32025 raft_consensus.cc:1081] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
W20260709 13:12:00.242949 32607 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:12:00.225657 32408 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.255977 32354 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:00.255154 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.236394 32626 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:00.243726 32552 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.677s	user 0.002s	sys 0.010s
I20260709 13:12:00.257436 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:00.261814 29078 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:00.241622 32623 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:00.245149 32423 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:00.254638 32628 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:00.256860 32622 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:12:00.273648 29708 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.243944 32467 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:00.271212 29084 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:00.268342 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.237968 32603 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.259739 32354 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.275452 32590 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.280432 29076 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:00.280956 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.263976 32282 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:00.279279 32604 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.261582 32626 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:00.299407 32408 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.302251 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.304407 32590 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.268540 32552 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.284672 32354 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:00.282721 32597 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:00.321170 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.313159 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.262581 32624 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.293798 32467 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:00.334342 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.335048 29002 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:00.280192 32464 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.337039 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.341104 29005 raft_consensus.cc:2410] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:12:00.343678 29713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94045d5a8709418e82919c9fbf804582"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:51170
I20260709 13:12:00.342999 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:00.345490 29007 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:00.329965 32560 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.344291 29713 raft_consensus.cc:606] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:00.367111 29708 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:00.328783 32355 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:00.356575 32411 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:00.369053 32355 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.369455 32571 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.371222 32355 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.372707 32603 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.346797 32522 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:00.374061 32522 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:00.374847 32522 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.346458 29085 raft_consensus.cc:2393] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:00.375777 32522 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.347458 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.379702 29002 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:12:00.380673 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:00.383013 32560 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.383857 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.385102 32560 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.388291 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:00.388571 29150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 443) took 1510 ms (client timeout 1943 ms). Trace:
W20260709 13:12:00.388952 29150 rpcz_store.cc:269] 0709 13:11:58.878518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.878673 (+   155us) service_pool.cc:224] Handling call
0709 13:12:00.388395 (+1509722us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54000942d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:00.388521 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1623133952}
I20260709 13:12:00.389904 32428 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.390405 32428 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.391561 29151 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:12:00.312534 32566 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:00.392740 32566 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:00.394001 32601 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.436s	user 0.000s	sys 0.000s
I20260709 13:12:00.395501 32566 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.322219 32611 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:00.396361 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1410) took 1193 ms. Trace:
I20260709 13:12:00.396184 32611 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.396718 29083 rpcz_store.cc:276] 0709 13:11:59.203339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.213040 (+  9701us) service_pool.cc:224] Handling call
0709 13:12:00.396140 (+1183100us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400404a54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:00.396273 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553789824}
I20260709 13:12:00.397975 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.398245 32611 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.398339 32552 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.400609 32428 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:00.401202 32428 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:00.401541 32428 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.408341 32428 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:00.410315 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1407) took 1210 ms. Trace:
I20260709 13:12:00.403173 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:00.343143 32597 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.343288 32467 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.377825 32464 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.377686 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.429821 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.430563 29085 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:00.431500 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.404750 32550 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:00.406113 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.407027 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.398644 29083 raft_consensus.cc:2368] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9.
I20260709 13:12:00.409337 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.349654 32624 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.412323 29158 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:00.414470 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.445981 29152 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:00.410697 29077 rpcz_store.cc:276] 0709 13:11:59.199523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.199650 (+   127us) service_pool.cc:224] Handling call
0709 13:12:00.410140 (+1210490us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54000c99d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:00.410256 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033167104}
I20260709 13:12:00.423216 29159 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:00.424629 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.428233 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:00.448565 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:00.441133 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.452451 29008 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:00.442283 29711 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:00.443177 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.453675 29153 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:00.444114 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.455574 29125 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:00.435004 29715 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:00.457085 29078 raft_consensus.cc:2410] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:00.457794 29712 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:00.457880 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.459167 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:00.459293 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.461050 29085 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:00.462499 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.446579 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.465122 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.465636 29154 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.465752 29150 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:00.467226 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.469774 29081 raft_consensus.cc:2368] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9.
I20260709 13:12:00.470510 32619 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:00.471596 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.517872 29159 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:00.518565 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.476298 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.462136 32590 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.477049 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.477306 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.478991 32574 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:00.479697 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:00.480140 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.433776 32408 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:12:00.480347 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.481925 32467 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.354053 32623 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.428027 32552 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:12:00.539255 32611 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:00.539644 32611 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:00.540026 32441 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.468235 32464 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:00.493337 29152 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:00.541371 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.542052 29152 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:00.542768 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.495451 29150 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:00.486178 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:00.545305 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.545464 29154 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:00.545907 29150 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
W20260709 13:12:00.498332 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:00.549053 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:00.485759 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.485128 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.506821 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.508858 29009 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:12:00.509792 29007 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:00.510715 29005 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:00.512706 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:00.555680 29008 raft_consensus.cc:2393] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:00.491019 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.405777 32628 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.522855 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.483991 32597 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:00.559378 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.527683 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.530339 29083 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:00.532555 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.560432 29081 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:00.560552 28976 connection.cc:256] RPC call timeout handler was delayed by -1.0965119070000355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
I20260709 13:12:00.560832 29153 raft_consensus.cc:2410] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:00.532701 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.533133 29710 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:00.518038 32584 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:12:00.534230 32563 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:00.582901 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:00.538625 32623 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:00.557112 29085 raft_consensus.cc:2410] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:12:00.558801 29716 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:00.569182 29152 raft_consensus.cc:2410] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:00.589166 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.585080 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.565657 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.589165 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.594516 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.594605 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:00.595922 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:00.597061 29159 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:00.598088 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:00.598783 29151 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:00.611404 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.559855 32552 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:00.659963 29714 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:00.583048 32563 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:00.665503 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.664441 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.738090 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:00.740099 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:00.712584 29076 raft_consensus.cc:2410] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:00.742345 29007 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:00.739588 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.743880 29083 raft_consensus.cc:2393] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:00.736274 29080 consensus_meta.cc:287] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.001s
I20260709 13:12:00.666625 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.745077 29080 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:00.747695 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.595526 32560 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:00.715858 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.665167 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:00.745803 29077 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:00.749579 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.755883 29085 raft_consensus.cc:2393] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:00.734552 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.724915 29081 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:00.759330 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 94255.5us
I20260709 13:12:00.762805 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.767590 29081 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.631560 32575 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.784220 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.787680 29711 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:00.784616 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.594901 32626 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:00.793852 32639 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 13:12:00.794274 32618 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.839s	user 0.000s	sys 0.003s
I20260709 13:12:00.784461 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.694244 32571 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:00.788645 29153 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:00.696695 32315 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.789199 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:00.706703 32590 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:00.736886 32584 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:00.758462 32625 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:00.780462 32546 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:00.886204 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.829631 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:00.783547 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 373318us
I20260709 13:12:00.836369 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:00.803680 29716 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:00.799018 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.827925 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:00.854586 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1412) took 1548 ms. Trace:
I20260709 13:12:00.811990 29081 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:00.855000 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.829968 29007 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:12:00.735998 32552 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.891264 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.892338 29081 raft_consensus.cc:2393] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:00.893388 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:00.888476 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 52194.1us
I20260709 13:12:00.897550 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.899732 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:00.900750 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 72892.2us
W20260709 13:12:00.901346 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.890134 29079 rpcz_store.cc:276] 0709 13:11:59.280946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.281126 (+   180us) service_pool.cc:224] Handling call
0709 13:11:59.355932 (+ 74806us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:59.538772 (+182840us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:59.538817 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:59.576248 (+ 37431us) log.cc:844] Serialized 26 byte log entry
0709 13:11:59.841366 (+265118us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:59.841482 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.841571 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:00.793614 (+952043us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:00.796536 (+  2922us) raft_consensus.cc:1685] finished
0709 13:12:00.796579 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.829165 (+ 32586us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":213382,"raft.queue_time_us":180830,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":54647,"spinlock_wait_cycles":7364864,"thread_start_us":201331,"threads_started":1,"wal-append.queue_time_us":302319}
I20260709 13:12:00.904268 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.900604 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:00.905633 29081 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:00.906420 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.906991 29009 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:00.891094 29007 pending_rounds.cc:85] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:12:00.912108 29007 pending_rounds.cc:107] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:12:00.912447 29007 raft_consensus.cc:2889] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:00.902547 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:00.922667 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:00.923817 29153 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:00.851953 32559 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.937093 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:00.931118 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:00.938309 29004 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:00.780987 32560 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.939889 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.939710 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.940858 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:00.932546 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.861727 32625 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.951642 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.944545 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:00.952760 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:00.950466 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.950830 29157 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:00.798290 32575 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.906059 32614 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:00.919816 32619 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:00.758307 32611 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.874588 32597 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:12:00.927538 32590 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:00.873080 32467 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:12:00.963761 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.011621 29083 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:00.975289 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:00.861121 32315 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:00.860769 32571 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:00.941334 32464 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:00.998004 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.026912 29007 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:01.009021 29081 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:00.966948 32614 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.921033 32584 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.012216 29009 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:01.020321 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:01.028249 29157 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
W20260709 13:12:01.030676 32644 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:12:01.034262 32315 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:01.036465 32571 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.089781 29156 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:01.050297 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.065976 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.068495 32467 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:01.068866 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.100504 29005 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:01.101531 29708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49414 (request call id 495) took 1936 ms (client timeout 2137 ms). Trace:
I20260709 13:12:00.962270 32625 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:01.075800 29709 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:00.898557 32563 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.110857 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.111557 29083 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:01.113178 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.056614 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.071048 32597 consensus_queue.cc:1243] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:01.044441 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.091542 29076 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:00.947337 32559 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.125320 29079 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:01.102993 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:01.102114 29708 rpcz_store.cc:269] 0709 13:11:59.139308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.139446 (+   138us) service_pool.cc:224] Handling call
0709 13:12:01.075107 (+1935661us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001f0754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:01.075264 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2606606080}
W20260709 13:12:01.086777 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.122442 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.128707 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.129127 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.123824 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 568) took 1319 ms. Trace:
I20260709 13:12:01.079402 32614 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:01.132206 29084 rpcz_store.cc:276] 0709 13:11:59.804922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.805067 (+   145us) service_pool.cc:224] Handling call
0709 13:11:59.805440 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.279699 (+474259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.123600 (+843901us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54001ad1d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:01.123774 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070667136}
I20260709 13:12:01.133360 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.134541 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:01.135046 29084 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:01.139053 32614 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.123282 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.141073 32556 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:01.147524 29084 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:01.173195 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.173499 29683 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:01.174101 29002 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:01.090973 32628 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:01.100062 32467 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:01.155828 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.145475 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:01.147754 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.182461 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.195006 32409 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.196859 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:01.197619 29710 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:01.199916 29076 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:01.095386 32355 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:12:01.202714 29077 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:12:01.204227 29077 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:01.205545 29077 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:01.195523 32409 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:01.189546 32647 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:12:01.175891 32565 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:01.213694 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.122854 32315 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 13:12:01.224629 29007 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:01.214706 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.183904 32510 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:01.209851 32355 consensus_queue.cc:1243] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:01.178534 32623 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:01.180505 32563 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:01.218559 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.178565 32596 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:12:01.226266 29150 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:01.226933 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.220012 29077 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:01.234166 29002 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:01.224974 32354 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.882s	user 0.005s	sys 0.005s
I20260709 13:12:01.240032 29077 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:12:01.241186 29713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.000s	sys 0.041s
W20260709 13:12:01.251861 29713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.000s	sys 0.041s
I20260709 13:12:01.253054 29077 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:01.183990 32556 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.256692 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.259440 29077 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:12:01.259855 29077 pending_rounds.cc:85] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 3
I20260709 13:12:01.260006 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:01.260234 29077 pending_rounds.cc:107] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:12:01.260600 29077 raft_consensus.cc:2889] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:01.232489 32626 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.261492 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.275005 29081 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:01.276207 29081 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:01.277452 29081 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:12:01.277842 29081 pending_rounds.cc:85] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 5
I20260709 13:12:01.278115 29081 pending_rounds.cc:107] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:12:01.278390 29081 raft_consensus.cc:2889] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:01.278708 29007 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:01.263175 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.284423 29009 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:01.235471 32510 raft_consensus.cc:697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:01.302947 29077 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:01.307747 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.252744 32552 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.230094 32565 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.262053 32614 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:01.338457 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:01.241335 32623 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.353305 32437 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.392830 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.289597 32467 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.403043 32467 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:01.404724 29076 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:01.405230 32623 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.405885 29085 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:01.366631 32423 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:12:01.291661 32630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.021s	sys 0.019s
W20260709 13:12:01.407776 32630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.021s	sys 0.019s
I20260709 13:12:01.389459 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.301982 32599 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.402583 32437 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.410884 32599 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.370683 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.403777 32565 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.413259 32437 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.251273 32354 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:01.394709 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.437289 29004 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:01.413301 32599 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:01.440944 32428 consensus_meta.cc:287] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.031s	user 0.011s	sys 0.001s
W20260709 13:12:01.441608 32550 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.995s	user 0.009s	sys 0.000s
I20260709 13:12:01.441424 32428 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.449630 29050 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:01.449798 32599 consensus_queue.cc:1059] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:01.450762 32623 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:01.451174 32623 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.451282 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:01.451586 32623 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:01.454795 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.455514 29001 raft_consensus.cc:2393] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:01.456471 29001 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:01.457219 32584 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:01.457899 32584 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:01.458295 32584 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:01.461180 29711 raft_consensus.cc:2368] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9.
I20260709 13:12:01.462366 32354 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:01.464604 32556 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:01.463697 32623 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.465449 32510 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.467281 32623 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.468146 32428 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:01.468986 32409 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.470171 32559 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.470450 29005 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:01.471306 29005 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
W20260709 13:12:01.473439 32649 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 13:12:01.474268 29153 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:12:01.474743 29082 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
W20260709 13:12:01.475334 32652 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.269s	user 0.000s	sys 0.000s
W20260709 13:12:01.475922 28973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.018s	sys 0.037s
W20260709 13:12:01.476293 28973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.018s	sys 0.037s
I20260709 13:12:01.476977 29150 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:01.480549 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:01.482308 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:01.494705 29050 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:01.495814 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:01.496625 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.497956 29004 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:12:01.498960 29004 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:01.459249 32584 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:01.502223 29125 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.014s	sys 0.052s
W20260709 13:12:01.504478 29125 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.014s	sys 0.052s
W20260709 13:12:01.504873 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 717695us
I20260709 13:12:01.504977 29122 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:01.502658 29050 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:01.506913 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:01.424337 32575 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.511739 29004 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:12:01.515134 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.476352 32624 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.503737 32623 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:01.501528 32409 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:01.407805 32423 consensus_queue.cc:1059] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
W20260709 13:12:01.522972 28976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.016s	sys 0.029s
I20260709 13:12:01.501705 32550 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:12:01.518416 29006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.013s	sys 0.031s
I20260709 13:12:01.503156 32556 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:01.510133 32565 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:01.528301 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.509533 29050 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:01.511042 32467 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:01.526654 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.500825 32354 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:12:01.518697 32599 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:01.525991 29006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.013s	sys 0.031s
I20260709 13:12:01.523161 32624 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:01.511427 32604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.010s	sys 0.042s
I20260709 13:12:01.531250 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:01.532889 29004 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:01.534976 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:01.529784 28976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.017s	sys 0.029s
W20260709 13:12:01.543617 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 597627us
I20260709 13:12:01.526755 32550 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.543659 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:01.543824 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 449) took 1160 ms. Trace:
I20260709 13:12:01.529948 32565 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.528527 32409 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:01.552601 32654 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:12:01.553825 29709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 506) took 1153 ms. Trace:
I20260709 13:12:01.554806 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1772) took 1367 ms. Trace:
I20260709 13:12:01.546631 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 520) took 1231 ms. Trace:
I20260709 13:12:01.549193 32574 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:01.555817 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:01.550765 32597 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:01.556241 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:01.557013 29004 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:01.593317 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:01.549324 32604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.010s	sys 0.042s
I20260709 13:12:01.559561 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.560899 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:01.561702 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2269) took 1091 ms. Trace:
I20260709 13:12:01.561853 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2273) took 1089 ms. Trace:
I20260709 13:12:01.563419 29125 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:01.591003 29006 rpcz_store.cc:276] 0709 13:12:00.306014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.308471 (+  2457us) service_pool.cc:224] Handling call
0709 13:12:00.329302 (+ 20831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.329572 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:00.519857 (+190285us) log.cc:844] Serialized 14 byte log entry
0709 13:12:01.536437 (+1016580us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:01.536489 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:01.536533 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.536595 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.537106 (+   511us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":187538,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":23,"raft.queue_time_us":109194,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7199,"thread_start_us":1193388,"threads_started":3,"wal-append.queue_time_us":603985}
W20260709 13:12:01.602978 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54996.8us
I20260709 13:12:01.604079 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1563) took 1115 ms. Trace:
I20260709 13:12:01.580945 32572 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:01.586163 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 511) took 1065 ms. Trace:
I20260709 13:12:01.531229 32584 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:01.605919 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:01.609967 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:01.602459 29158 rpcz_store.cc:276] 0709 13:12:00.470538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.475760 (+  5222us) service_pool.cc:224] Handling call
0709 13:12:00.476161 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.476604 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.476642 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.700420 (+223778us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b2c0012c0f8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:12:00.700807 (+   387us) log.cc:844] Serialized 26 byte log entry
0709 13:12:01.075672 (+374865us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:01.075725 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.075807 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.561352 (+485545us) raft_consensus.cc:1685] finished
0709 13:12:01.561378 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.561634 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":81,"spinlock_wait_cycles":50346880,"thread_start_us":374483,"threads_started":1,"wal-append.queue_time_us":791595}
I20260709 13:12:01.567798 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:01.604698 29159 rpcz_store.cc:276] 0709 13:12:00.522141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.560583 (+1038442us) service_pool.cc:224] Handling call
0709 13:12:01.603980 (+ 43397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:01.613879 29009 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:01.579216 32611 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:01.616273 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.617076 29079 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:01.540531 32626 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:01.618757 32626 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:01.620127 28974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.016s	sys 0.024s
I20260709 13:12:01.619973 32626 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:01.620468 28974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.016s	sys 0.024s
I20260709 13:12:01.603991 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:01.624575 32619 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:01.625069 29125 leader_election.cc:400] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9.
I20260709 13:12:01.625093 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.625649 29125 leader_election.cc:403] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:12:01.590831 29082 rpcz_store.cc:276] 0709 13:12:00.187754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.187895 (+   141us) service_pool.cc:224] Handling call
0709 13:12:01.554669 (+1366774us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540025ab54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:01.554755 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2829976448}
W20260709 13:12:01.622349 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 54654.1us
I20260709 13:12:01.589763 29709 rpcz_store.cc:276] 0709 13:12:00.400698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.401798 (+  1100us) service_pool.cc:224] Handling call
0709 13:12:01.075387 (+673589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.108449 (+ 33062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.553644 (+445195us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b54001f0750. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:01.553769 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548307712}
I20260709 13:12:01.588922 29155 rpcz_store.cc:276] 0709 13:12:00.383876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.384129 (+   253us) service_pool.cc:224] Handling call
0709 13:12:00.384439 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.464035 (+ 79596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.464098 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:00.464172 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:00.464218 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.543618 (+1079400us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b540013d654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:01.543662 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.543774 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182467968}
W20260709 13:12:01.587644 29003 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.030s	sys 0.024s
W20260709 13:12:01.629087 29003 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.030s	sys 0.025s
I20260709 13:12:01.629909 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 525) took 1322 ms. Trace:
I20260709 13:12:01.630312 29003 rpcz_store.cc:276] 0709 13:12:00.307397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.307517 (+   120us) service_pool.cc:224] Handling call
0709 13:12:00.307794 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.307983 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:00.603465 (+295482us) log.cc:844] Serialized 14 byte log entry
0709 13:12:01.629494 (+1026029us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:01.629562 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:01.629619 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.629704 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.629845 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":208727,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":156049,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":52831,"thread_start_us":1261281,"threads_started":3,"wal-append.queue_time_us":567423}
I20260709 13:12:01.631669 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.632597 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.635422 29080 pending_rounds.cc:85] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 3
I20260709 13:12:01.641674 29080 pending_rounds.cc:107] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:01.640174 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:01.587031 32565 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:01.612563 29152 rpcz_store.cc:276] 0709 13:12:00.471968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.542566 (+ 70598us) service_pool.cc:224] Handling call
0709 13:12:01.561625 (+1019059us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b54002a44d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:01.561786 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58601728}
I20260709 13:12:01.590399 32550 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:01.641645 29079 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:01.642652 29082 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:01.642863 29076 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:01.605425 29713 rpcz_store.cc:276] 0709 13:12:00.520563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.544801 (+ 24238us) service_pool.cc:224] Handling call
0709 13:12:00.599605 (+ 54804us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:00.709350 (+109745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.709384 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:00.709970 (+   586us) log.cc:844] Serialized 52 byte log entry
0709 13:12:01.252351 (+542381us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:01.252388 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.252464 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.585924 (+333460us) raft_consensus.cc:1685] finished
0709 13:12:01.585947 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.586081 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99817,"raft.queue_time_us":110513,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11285,"thread_start_us":542026,"threads_started":1,"wal-append.queue_time_us":826182}
I20260709 13:12:01.644371 29006 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:01.551070 32467 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:01.642094 29080 raft_consensus.cc:2889] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:01.648945 29081 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:01.598551 32597 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.608089 32566 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:01.663494 29002 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:01.608546 32572 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:01.640739 32464 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:01.656153 29682 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.006s	sys 0.034s
I20260709 13:12:01.640894 32355 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:01.612151 32575 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:01.662739 32597 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:01.668617 32572 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.647480 32546 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:12:01.669310 32467 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.669934 32409 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.672324 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.677892 32550 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:01.644747 32556 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:01.688860 32565 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:01.738723 29048 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.014s	sys 0.028s
W20260709 13:12:01.739058 29048 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.014s	sys 0.028s
I20260709 13:12:01.695616 32464 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:01.641145 32571 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:01.692010 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:01.693269 32560 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:01.694694 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1558) took 1207 ms. Trace:
I20260709 13:12:01.746999 29156 rpcz_store.cc:276] 0709 13:12:00.487305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.076692 (+589387us) service_pool.cc:224] Handling call
0709 13:12:01.082577 (+  5885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.105101 (+ 22524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.694408 (+589307us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b5400054e50. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:01.694594 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12704128}
W20260709 13:12:01.695920 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.696592 29682 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.006s	sys 0.034s
W20260709 13:12:01.748492 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.748548 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 684833us
W20260709 13:12:01.749513 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica befda4a6c8344c26b0f139e4e1686b3b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:01.700906 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:01.711485 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.714404 32355 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:01.662305 32574 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9.
I20260709 13:12:01.729172 29008 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:01.734913 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.741282 29010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.036s	sys 0.060s
I20260709 13:12:01.743767 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.690008 32611 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.745821 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:01.754170 29010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.036s	sys 0.060s
W20260709 13:12:01.754995 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 54213us
W20260709 13:12:01.755067 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.761533 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.809217 29156 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:01.809376 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 50689.9us
I20260709 13:12:01.809971 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1971) took 1303 ms. Trace:
I20260709 13:12:01.810477 29008 rpcz_store.cc:276] 0709 13:12:00.506077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.658265 (+152188us) service_pool.cc:224] Handling call
0709 13:12:01.728759 (+1070494us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:01.728992 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.756118 (+ 27126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.809915 (+ 53797us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75015936}
I20260709 13:12:01.811177 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:01.729642 32568 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:12:01.753523 29717 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.014s	sys 0.019s
I20260709 13:12:01.812479 29009 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:01.736406 32408 raft_consensus.cc:3055] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:01.817843 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:01.814183 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 531) took 1453 ms. Trace:
W20260709 13:12:01.812577 29717 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.014s	sys 0.019s
I20260709 13:12:01.751040 32464 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:01.969110 29084 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.042s	sys 0.031s
W20260709 13:12:01.969563 29084 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.042s	sys 0.031s
W20260709 13:12:01.835407 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 82909.7us
I20260709 13:12:01.841768 29712 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:01.842303 29708 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:01.832630 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:01.826719 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:01.870365 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.973836 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:01.974261 29008 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:12:01.873592 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.975301 29156 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:01.832080 29010 rpcz_store.cc:276] 0709 13:12:00.360512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.360670 (+   158us) service_pool.cc:224] Handling call
0709 13:12:00.361046 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.361302 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:00.604322 (+243020us) log.cc:844] Serialized 14 byte log entry
0709 13:12:01.762378 (+1158056us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:01.813797 (+ 51419us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:01.813871 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.813959 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.814114 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25495,"prepare.queue_time_us":155996,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":204639,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":9814,"thread_start_us":1364239,"threads_started":3,"wal-append.queue_time_us":670365}
W20260709 13:12:01.976352 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 149736us
I20260709 13:12:01.976711 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.976809 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:01.977095 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:01.966892 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:01.978092 29004 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:12:01.981420 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:01.752315 32556 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:01.980994 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:01.982625 29714 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:01.811401 32563 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.989352 32564 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:01.989754 29713 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:01.989490 32563 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:01.811648 32571 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:01.829944 32664 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:01.987368 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:01.980983 32623 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:01.988196 32354 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.004743 32614 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:02.004849 32354 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.006958 32354 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:02.007439 32354 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.007851 32354 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.009696 32354 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:01.988539 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:02.010331 29156 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:02.011237 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:01.988474 32672 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:01.984714 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:01.989378 32599 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.017782 32563 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:01.990131 32564 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.020514 32564 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:02.021231 29151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.042s	sys 0.029s
I20260709 13:12:01.985105 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:02.026300 29048 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:01.830101 32560 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:02.029615 32564 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:02.029901 32560 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:02.030321 32560 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:02.000989 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1780) took 1787 ms. Trace:
I20260709 13:12:01.813247 32568 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:01.830107 32408 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:01.966562 32574 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:02.027624 29151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.042s	sys 0.030s
I20260709 13:12:01.811297 32596 consensus_queue.cc:1059] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.181s
I20260709 13:12:01.815124 32611 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.033245 29085 rpcz_store.cc:276] 0709 13:12:00.213610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.147522 (+933912us) service_pool.cc:224] Handling call
0709 13:12:02.000831 (+853309us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540027b0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:02.000936 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502673664}
I20260709 13:12:02.039346 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:01.984472 32025 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:02.048146 29048 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:02.050549 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:02.024214 32664 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:02.042352 32625 consensus_meta.cc:287] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.911s	user 0.011s	sys 0.000s
I20260709 13:12:02.037163 32592 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:02.059052 29683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.031s	sys 0.046s
W20260709 13:12:02.061127 29683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.031s	sys 0.046s
W20260709 13:12:02.061789 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 654263us
I20260709 13:12:02.011808 29004 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:01.995959 29154 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:02.076465 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2285) took 1600 ms. Trace:
I20260709 13:12:02.079557 29008 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
W20260709 13:12:02.084288 29001 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.002s	sys 0.024s
W20260709 13:12:02.088860 29001 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.002s	sys 0.024s
W20260709 13:12:02.084798 29083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.037s	sys 0.028s
W20260709 13:12:02.089586 29083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.037s	sys 0.028s
I20260709 13:12:02.046634 32623 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:02.077900 29154 rpcz_store.cc:276] 0709 13:12:00.486302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.691725 (+1205423us) service_pool.cc:224] Handling call
0709 13:12:02.076392 (+384667us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:02.096251 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 285119us
I20260709 13:12:02.153386 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:02.154162 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:02.053519 32596 raft_consensus.cc:3055] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:02.108541 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 1984) took 1501 ms. Trace:
W20260709 13:12:02.155072 29125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.021s	sys 0.014s
I20260709 13:12:02.112710 32666 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:02.052618 32355 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.030331 32564 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.090513 32546 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.095661 32550 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:02.054211 32025 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.138597 32563 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:02.063894 32625 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:02.139357 32662 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
W20260709 13:12:02.155435 29125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.021s	sys 0.014s
I20260709 13:12:02.109082 29005 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:02.178094 29005 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:02.178157 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 553164us
W20260709 13:12:02.148746 32559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.006s	sys 0.022s
I20260709 13:12:02.155237 29001 rpcz_store.cc:276] 0709 13:12:00.607594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.457085 (+849491us) service_pool.cc:224] Handling call
0709 13:12:01.457304 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.457564 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:01.457826 (+   262us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.108091 (+650265us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:02.108163 (+    72us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:02.108226 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.108321 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.108473 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18501,"prepare.queue_time_us":2774,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":19614,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":13445,"thread_start_us":631299,"threads_started":1,"wal-append.queue_time_us":682009}
I20260709 13:12:02.158601 32596 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:02.187465 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 207182us
I20260709 13:12:02.125758 32671 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:02.048209 32574 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.171461 29150 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:02.190531 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:02.174425 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:02.154265 32661 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.006s	sys 0.022s
I20260709 13:12:02.191670 29008 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
W20260709 13:12:02.190425 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 216642us
I20260709 13:12:02.175088 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:02.195969 29154 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:02.173362 32623 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:02.105813 32408 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.179627 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:02.181918 32486 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.013s	sys 0.030s
I20260709 13:12:02.202972 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:02.181488 29009 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
W20260709 13:12:02.183661 29082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.010s	sys 0.014s
W20260709 13:12:02.183919 29002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.005s	sys 0.019s
W20260709 13:12:02.203953 29082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.010s	sys 0.014s
I20260709 13:12:02.113215 32556 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.204782 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:02.187539 29717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 733) took 1359 ms. Trace:
W20260709 13:12:02.169101 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.187868 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:02.206640 29001 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
W20260709 13:12:02.186187 32604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.008s
I20260709 13:12:02.214236 29156 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:02.118970 32571 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.196249 29078 raft_consensus.cc:2368] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11.
I20260709 13:12:02.151791 32565 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:02.105046 32568 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.198086 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:02.199162 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 148686us
I20260709 13:12:02.221181 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:02.221784 29008 raft_consensus.cc:2410] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:12:02.221524 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:02.199834 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:02.200006 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:02.188956 32559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.006s	sys 0.022s
I20260709 13:12:02.223361 32411 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:02.170341 32546 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:02.226466 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 122053us
I20260709 13:12:02.164740 32665 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:02.207254 29150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:02.212666 29125 leader_election.cc:400] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9.
I20260709 13:12:02.211081 29005 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.140300 32664 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.215180 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:02.189702 32671 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.217345 32354 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:12:02.258912 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.216699 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:02.193218 32672 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:02.217864 29716 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
I20260709 13:12:02.262269 29076 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:02.201370 32486 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.013s	sys 0.030s
I20260709 13:12:02.222517 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:02.280072 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:02.200839 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:02.280853 29081 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:02.281486 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 80770.3us
I20260709 13:12:02.223953 32411 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:02.207201 29717 rpcz_store.cc:276] 0709 13:12:00.828382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.000715 (+172333us) service_pool.cc:224] Handling call
0709 13:12:01.026005 (+ 25290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.026243 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:01.833722 (+807479us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.187214 (+353492us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:02.187266 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:02.187303 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.187364 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.187477 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4005,"prepare.queue_time_us":1129655,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":2560,"raft.queue_time_us":9161,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":2247,"thread_start_us":1145584,"threads_started":2}
W20260709 13:12:02.204012 29002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.005s	sys 0.019s
I20260709 13:12:02.285223 29079 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:02.222894 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:02.206714 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:02.287132 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 80481.3us
I20260709 13:12:02.288302 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:02.228972 29125 leader_election.cc:403] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:02.214371 32604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.007s	sys 0.009s
I20260709 13:12:02.230290 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1792) took 1957 ms. Trace:
I20260709 13:12:02.213748 32681 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:02.243770 29001 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:02.231671 32568 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:02.292769 29079 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:12:02.293141 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:02.195997 32661 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.006s	sys 0.022s
W20260709 13:12:02.248139 32686 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:12:02.282563 32674 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:12:02.259230 32354 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.252527 32692 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:02.282434 32664 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:02.254256 32574 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:02.262698 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:02.264519 29076 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:02.266319 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.282407 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:02.222865 29154 raft_consensus.cc:2410] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:12:02.289556 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 83478.8us
I20260709 13:12:02.313010 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:02.289254 29084 rpcz_store.cc:276] 0709 13:12:00.271690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.174500 (+902810us) service_pool.cc:224] Handling call
0709 13:12:01.174756 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.174949 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:01.969990 (+795041us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.228234 (+258244us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:02.228282 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:02.228327 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.228396 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.228506 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1820,"prepare.queue_time_us":79774,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":33041,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"thread_start_us":996897,"threads_started":2}
I20260709 13:12:02.231366 32611 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:02.297864 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 75032.1us
I20260709 13:12:02.316658 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1793) took 2042 ms. Trace:
I20260709 13:12:02.317054 29083 rpcz_store.cc:276] 0709 13:12:00.275253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.177686 (+902433us) service_pool.cc:224] Handling call
0709 13:12:01.207589 (+ 29903us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:01.207968 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:01.208002 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:01.215342 (+  7340us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.149555 (+934213us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:02.190794 (+ 41239us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.190928 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.191231 (+   303us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.316451 (+125220us) raft_consensus.cc:1685] finished
0709 13:12:02.316479 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.316606 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41054,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":52756,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":32708,"thread_start_us":886601,"threads_started":1,"wal-append.queue_time_us":886833}
I20260709 13:12:02.294263 28976 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:02.250330 32572 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:02.320206 32676 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:02.320736 32676 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:02.321208 32546 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:02.321674 32676 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:02.324460 32675 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:12:02.297864 32665 raft_consensus.cc:2764] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:02.259778 32671 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:02.304466 32690 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:02.304289 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 82842.4us
I20260709 13:12:02.309791 32354 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:02.251811 32025 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:02.326454 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:02.327332 29081 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:02.300254 32559 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:02.295445 32467 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9.
W20260709 13:12:02.328274 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.300954 32603 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:02.303956 32592 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:02.334565 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:02.339051 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:02.345212 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:02.231211 32596 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.329797 32611 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:02.331561 32574 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:12:02.350523 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.355087 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1566) took 1563 ms. Trace:
I20260709 13:12:02.370756 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:02.372520 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55310.2us
I20260709 13:12:02.335712 32315 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:12:02.346307 32669 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:12:02.376582 32315 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:12:02.348528 32677 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:12:02.378753 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:02.362078 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:02.365689 29151 rpcz_store.cc:276] 0709 13:12:00.791261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.262331 (+471070us) service_pool.cc:224] Handling call
0709 13:12:01.350303 (+ 87972us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:01.400442 (+ 50139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:01.400489 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:01.401141 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.036417 (+635276us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:02.036479 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.036576 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.354879 (+318303us) raft_consensus.cc:1685] finished
0709 13:12:02.354899 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.355029 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17444,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":121,"thread_start_us":634786,"threads_started":1,"wal-append.queue_time_us":754697}
I20260709 13:12:02.356309 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:02.381791 29083 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:02.377007 32669 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:02.382300 32592 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:02.382434 32025 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:02.383359 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1568) took 1445 ms. Trace:
I20260709 13:12:02.383263 32592 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.383737 29151 rpcz_store.cc:276] 0709 13:12:00.938378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.382545 (+1444167us) service_pool.cc:224] Handling call
0709 13:12:02.382879 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.383058 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.383083 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:02.383123 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:02.383154 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.383204 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.383310 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:02.347122 32681 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:02.347851 32559 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.348670 32603 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.332926 32666 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 13:12:02.395185 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 69015.3us
W20260709 13:12:02.396796 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 72018.1us
I20260709 13:12:02.364291 32611 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.344364 32546 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:12:02.368930 32564 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:02.391813 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.393163 32522 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:02.419852 29048 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:02.420781 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:02.429342 32692 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:02.429587 29048 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:02.430852 32565 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:02.429746 32692 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:02.431737 32692 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:02.432667 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:02.434826 32565 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:02.400696 32559 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:02.450891 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.386564 32467 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:02.379674 32409 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:02.407079 32611 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:02.382318 32669 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:02.377010 32315 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:02.391959 32625 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:02.394096 32596 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.467939 29010 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:02.463747 32584 consensus_meta.cc:287] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.851s	user 0.006s	sys 0.006s
I20260709 13:12:02.393968 32700 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:02.497949 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 89838.9us
I20260709 13:12:02.404997 32603 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:02.351327 32411 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.506727 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2279) took 2033 ms. Trace:
W20260709 13:12:02.513803 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 124927us
I20260709 13:12:02.521225 32669 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.490749 32467 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:02.463305 32681 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:12:02.526214 29150 raft_consensus.cc:606] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:02.466269 32705 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:12:02.507126 29153 rpcz_store.cc:276] 0709 13:12:00.473784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.932155 (+458371us) service_pool.cc:224] Handling call
0709 13:12:00.953276 (+ 21121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:01.474734 (+521458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.506480 (+1031746us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54003f66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:02.506671 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197382016}
I20260709 13:12:02.469995 32693 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:02.555938 29153 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:02.549783 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51128.4us
I20260709 13:12:02.495293 32564 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.495505 32355 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:02.555876 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:02.495268 32680 log.cc:927] Time spent T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:12:02.568172 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1572) took 1442 ms. Trace:
W20260709 13:12:02.550046 32694 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:12:02.555074 32682 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 13:12:02.554737 32697 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:12:02.568598 29153 rpcz_store.cc:276] 0709 13:12:01.126102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.554707 (+1428605us) service_pool.cc:224] Handling call
0709 13:12:02.555051 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.567920 (+ 12869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.568116 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:02.586416 29153 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:02.587042 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 456) took 1099 ms. Trace:
I20260709 13:12:02.546990 32628 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:02.564256 32693 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.587555 29153 rpcz_store.cc:276] 0709 13:12:01.487562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.585766 (+1098204us) service_pool.cc:224] Handling call
0709 13:12:02.586079 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.586765 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.586971 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:02.593668 29153 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:02.589066 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:02.584617 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 244482us
I20260709 13:12:02.566270 32355 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:02.580235 32669 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:02.596912 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2286) took 1109 ms. Trace:
I20260709 13:12:02.608420 29153 rpcz_store.cc:276] 0709 13:12:01.487844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.593011 (+1105167us) service_pool.cc:224] Handling call
0709 13:12:02.593366 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.596685 (+  3319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.596861 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:02.605014 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.607508 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:02.612141 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.613067 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:02.617888 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:02.618667 29009 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:02.591601 32486 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.589781 32628 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:02.568830 32564 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.584393 32690 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:02.595630 32693 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:02.652406 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:02.644416 29002 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:12:02.624390 32522 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.580449 32315 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:02.636056 32464 consensus_meta.cc:287] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.000s
I20260709 13:12:02.624691 32712 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.603281 32665 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:12:02.607960 32666 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:12:02.720359 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 107496us
I20260709 13:12:02.603133 32546 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 13:12:02.705257 32682 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:12:02.705502 32698 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
W20260709 13:12:02.705428 32688 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
I20260709 13:12:02.596335 32711 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:12:02.709841 32409 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:02.719712 32692 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.740775 32681 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:12:02.714294 32522 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.745994 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1567) took 1860 ms. Trace:
W20260709 13:12:02.809849 32560 consensus_meta.cc:287] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.003s
I20260709 13:12:02.831415 32560 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.831753 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:02.833406 32560 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:02.755306 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 198025us
W20260709 13:12:02.835024 32703 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:12:02.705576 32716 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:02.836433 32709 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:12:02.836380 32716 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:02.837970 32710 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:12:02.838783 32716 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:02.839916 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:02.841142 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:02.741997 32599 consensus_meta.cc:287] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.729s	user 0.007s	sys 0.004s
W20260709 13:12:02.760695 28975 connection.cc:256] RPC call timeout handler was delayed by -1.2704794390000416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 13:12:02.843600 28975 connection.cc:256] RPC call timeout handler was delayed by -1.2542794390000154s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 13:12:02.645164 32486 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.741927 32464 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.704907 32715 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:02.826195 29154 rpcz_store.cc:276] 0709 13:12:00.885538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.319720 (+1434182us) service_pool.cc:224] Handling call
0709 13:12:02.325660 (+  5940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.326019 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.326047 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.326452 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.495719 (+169267us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:02.495774 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.495864 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.745738 (+249874us) raft_consensus.cc:1685] finished
0709 13:12:02.745764 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.745924 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":642,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4866,"thread_start_us":141007,"threads_started":1,"wal-append.queue_time_us":400515}
I20260709 13:12:02.705263 32696 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.747205 32712 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.828085 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1562) took 2339 ms. Trace:
I20260709 13:12:02.862991 29157 rpcz_store.cc:276] 0709 13:12:00.488509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.082864 (+594355us) service_pool.cc:224] Handling call
0709 13:12:01.083462 (+   598us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:01.141113 (+ 57651us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:01.184309 (+ 43196us) log.cc:844] Serialized 14 byte log entry
0709 13:12:01.571670 (+387361us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800384160. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112754557 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:12:02.093118 (+521448us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b1800384160. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:12:02.306681 (+213563us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:02.307436 (+   755us) log.cc:844] Serialized 52 byte log entry
0709 13:12:02.307600 (+   164us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:02.307647 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.307735 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.827792 (+520057us) raft_consensus.cc:1685] finished
0709 13:12:02.827820 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.827993 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33783,"prepare.queue_time_us":47248,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2443,"raft.queue_time_us":62805,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":8404,"spinlock_wait_cycles":45143168,"thread_start_us":213228,"threads_started":1,"wal-append.queue_time_us":1158643}
I20260709 13:12:02.748437 32355 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.757822 32628 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.830026 32690 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.821743 32559 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.705933 32467 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:02.845002 28975 connection.cc:256] RPC call timeout handler was delayed by -1.0814794389999633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
I20260709 13:12:02.847577 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:02.873756 28975 connection.cc:256] RPC call timeout handler was delayed by -1.0634794390000479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:12:02.874753 28975 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:02.852797 29711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 461) took 2067 ms (client timeout 1697 ms). Trace:
I20260709 13:12:02.861747 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:02.875852 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:02.876461 29004 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:02.851972 32664 consensus_meta.cc:287] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.526s	user 0.002s	sys 0.007s
I20260709 13:12:02.868273 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:02.880491 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 132764us
I20260709 13:12:02.849622 32565 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.850668 32672 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.852058 32706 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:02.883569 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:02.891705 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.853842 32599 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:02.876264 29711 rpcz_store.cc:269] 0709 13:12:00.785198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.795884 (+ 10686us) service_pool.cc:224] Handling call
0709 13:12:02.852606 (+2056722us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54004b7654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:02.852731 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311069440}
I20260709 13:12:02.858036 32699 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:02.890677 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:02.892889 29154 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:12:02.903770 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:02.904878 29085 raft_consensus.cc:2393] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:02.861191 32556 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.914194 29004 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:02.916848 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:02.920843 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:02.866304 32423 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:12:02.823733 32409 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.850214 32722 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:12:02.940267 32628 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:02.869828 32696 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:02.940618 32692 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:02.928351 32599 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:02.941385 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2293) took 1362 ms. Trace:
I20260709 13:12:02.875217 32723 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:12:02.954901 29156 rpcz_store.cc:276] 0709 13:12:01.579138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.010958 (+431820us) service_pool.cc:224] Handling call
0709 13:12:02.941217 (+930259us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54003ecbd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:02.941327 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465529728}
I20260709 13:12:02.942378 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:02.956539 29152 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:02.957583 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:02.958828 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:02.959417 29152 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:02.960481 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:02.961155 29152 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:02.947321 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 71549.4us
W20260709 13:12:02.921777 32671 consensus_meta.cc:287] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.004s
I20260709 13:12:02.919271 32724 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:02.965745 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:02.933236 32464 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:02.920117 32355 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:02.928573 32354 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.002s
I20260709 13:12:02.916915 32560 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:02.924784 32664 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:02.946028 32564 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:02.902894 32565 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.980955 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:02.921262 32556 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:02.988844 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 87473.7us
I20260709 13:12:02.989387 29156 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:02.867431 32715 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:02.992000 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:02.995183 32464 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:02.996628 32564 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:02.997459 29159 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:02.998179 32564 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:02.903095 32672 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.000865 29156 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:03.001920 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.002460 29156 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:03.002681 32672 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.003283 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.005082 29715 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:02.993566 29050 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:03.012212 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2297) took 1338 ms. Trace:
I20260709 13:12:03.015702 29156 rpcz_store.cc:276] 0709 13:12:01.674329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.011335 (+1337006us) service_pool.cc:224] Handling call
0709 13:12:03.011698 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:03.011869 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:03.011899 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:03.011941 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:03.011974 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.012028 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.012156 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:03.037091 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44198us
W20260709 13:12:03.037923 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44444.6us
I20260709 13:12:02.973811 32724 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:03.041162 28976 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.66:35939 timed out after 2.130s (SENT) [suppressed 8 similar messages]
I20260709 13:12:03.042136 29152 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:03.045389 29152 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:12:03.046808 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:03.047425 29152 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:12:03.048650 29152 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:03.048810 29083 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:12:03.050005 29152 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:03.038868 29050 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:03.051990 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:03.057652 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 197911us
I20260709 13:12:02.988844 32666 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:03.062669 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 458) took 1311 ms. Trace:
I20260709 13:12:03.068507 29156 rpcz_store.cc:276] 0709 13:12:01.751177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.020484 (+1269307us) service_pool.cc:224] Handling call
0709 13:12:03.061929 (+ 41445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:03.062218 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:03.062256 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:03.062333 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:03.062380 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.062451 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.062587 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:03.063360 29085 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:03.064913 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:02.967432 32611 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.012219 32559 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:02.972802 32599 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:03.073060 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 81179.2us
I20260709 13:12:03.076009 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:03.044804 32672 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:02.979930 32354 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.077696 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:03.040078 32724 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:02.976603 32671 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:03.071553 32626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.448s	user 0.010s	sys 0.013s
I20260709 13:12:03.043380 32664 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:03.079082 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:03.091131 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:03.070220 32692 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:03.066540 32604 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:03.083033 29156 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:03.089677 29159 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:12:03.092343 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:03.077674 32571 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:03.090339 32626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.467s	user 0.010s	sys 0.013s
I20260709 13:12:03.097620 32672 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:03.117996 29079 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:03.083839 32732 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:03.119465 32732 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:03.121085 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.120893 32604 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:03.122822 29050 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:03.082999 32734 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:03.124924 32604 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.095000 32692 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:03.097020 32625 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:03.127169 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:03.137730 29007 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:03.128018 29151 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:03.129316 29082 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:03.012465 32560 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.040093 32727 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:12:03.083292 32699 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:03.142661 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 65199.6us
I20260709 13:12:03.143452 28975 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:03.127929 32486 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:03.148675 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 70133.3us
W20260709 13:12:03.134169 32025 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.751s	user 0.009s	sys 0.002s
I20260709 13:12:03.150765 32212 raft_consensus.cc:3055] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:03.150839 32611 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.151787 32731 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:03.128190 32664 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.152756 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 538) took 1423 ms. Trace:
I20260709 13:12:03.153110 29010 rpcz_store.cc:276] 0709 13:12:01.729660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.987268 (+257608us) service_pool.cc:224] Handling call
0709 13:12:01.987553 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.495019 (+507466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.152540 (+657521us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b5400026554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:03.152696 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104188800}
I20260709 13:12:03.153904 32664 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:03.154350 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:03.155423 32568 consensus_meta.cc:287] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.856s	user 0.005s	sys 0.005s
I20260709 13:12:03.155459 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:03.156394 32713 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.330s	user 0.002s	sys 0.000s
I20260709 13:12:03.156551 32568 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:03.156775 32212 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:03.128530 32522 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:03.157794 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:03.157145 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.158304 29010 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:03.149798 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.134065 32672 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:03.153847 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:03.151093 32729 log.cc:927] Time spent T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:12:03.132278 32732 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:03.146091 32597 consensus_meta.cc:287] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.420s	user 0.014s	sys 0.000s
I20260709 13:12:03.136715 32692 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:03.174324 32737 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:12:03.161710 32025 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:03.176177 32409 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.178150 28975 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.161682 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:03.164776 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:03.179129 29010 raft_consensus.cc:2368] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10.
I20260709 13:12:03.180083 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:03.134248 32626 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:03.130722 32559 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:12:03.159880 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:03.181699 28975 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.161370 32522 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:03.186287 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:03.157178 32568 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.183315 32666 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:03.192378 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:03.192734 32739 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:03.193387 29001 raft_consensus.cc:2410] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:12:03.181036 32546 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:03.194216 32715 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.151183 32741 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:12:03.196911 32571 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:03.157701 32212 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.197605 32212 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:03.197814 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.198536 29152 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 10: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:03.199878 28975 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.201069 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:03.177484 32510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.646s	user 0.015s	sys 0.013s
I20260709 13:12:03.147550 32699 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.163337 32712 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:03.207281 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.176959 32409 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
W20260709 13:12:03.212589 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52802.8us
I20260709 13:12:03.213516 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:03.209640 29010 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:12:03.214198 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:03.194808 32738 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.187402 32626 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:03.208534 32510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.679s	user 0.016s	sys 0.013s
I20260709 13:12:03.192444 32568 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:03.212915 32696 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.261289 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:03.261186 32739 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:03.181143 32597 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.260349 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:03.262326 29156 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:12:03.216301 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:03.210903 32666 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:03.214128 32590 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:03.212975 32571 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:03.269281 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:03.262979 32025 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:03.212980 32711 consensus_queue.cc:1243] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:03.212862 32731 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:03.265187 32625 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:03.274115 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:03.256354 32734 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:03.283449 32696 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:03.271267 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:03.258427 32626 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:03.281417 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.281890 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:03.282549 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.257732 32565 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:03.284430 32749 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:03.265031 32409 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:03.287966 29155 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:03.288918 32742 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:12:03.268236 32556 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.208956 32664 consensus_queue.cc:1059] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:12:03.296728 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:03.309674 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:03.322826 32748 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:12:03.323496 29152 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:03.308537 29156 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:03.270900 32737 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:12:03.283061 32666 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:03.329603 32693 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.002s
I20260709 13:12:03.287305 32025 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:03.330200 32025 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:03.330039 32693 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.330466 29152 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:03.320079 32597 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:03.329182 32666 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.332589 32664 raft_consensus.cc:3055] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:03.332984 32664 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:03.292809 32603 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.743s	user 0.005s	sys 0.010s
I20260709 13:12:03.334285 32603 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.258231 32738 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.336491 29001 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:03.337939 32738 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:03.339157 32699 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.296558 32564 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:12:03.310081 32671 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.312718 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:03.340966 29078 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1418) took 2789 ms (client timeout 2130 ms). Trace:
W20260709 13:12:03.341531 29078 rpcz_store.cc:269] 0709 13:12:00.551120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.897369 (+346249us) service_pool.cc:224] Handling call
0709 13:12:03.340669 (+2443300us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54000460d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:03.340840 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2893488896}
I20260709 13:12:03.346361 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.347048 29154 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:03.353824 32624 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:03.356389 32669 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.730s	user 0.010s	sys 0.000s
I20260709 13:12:03.359007 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1569) took 2390 ms. Trace:
I20260709 13:12:03.359395 29151 rpcz_store.cc:276] 0709 13:12:00.968519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.384659 (+1416140us) service_pool.cc:224] Handling call
0709 13:12:02.384997 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:03.138375 (+753378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.358768 (+220393us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b5400055354. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:03.358951 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1708125440}
I20260709 13:12:03.360244 29010 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:03.368680 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 86165.9us
I20260709 13:12:03.370280 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.293246 32590 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:03.292634 32571 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:03.298008 32696 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:03.374199 32025 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.372638 32625 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.377437 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:03.326591 32596 consensus_meta.cc:287] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.830s	user 0.006s	sys 0.003s
I20260709 13:12:03.399053 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:03.395133 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 241331us
I20260709 13:12:03.399698 29711 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
W20260709 13:12:03.395735 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 259691us
I20260709 13:12:03.396871 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:03.404439 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:03.405457 29152 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:03.408335 29154 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:03.410405 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2296) took 1754 ms. Trace:
I20260709 13:12:03.411034 29153 rpcz_store.cc:276] 0709 13:12:01.655978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.614567 (+958589us) service_pool.cc:224] Handling call
0709 13:12:02.614838 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.615225 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:02.708345 (+ 93120us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.200111 (+491766us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b180040ada0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:12:03.398793 (+198682us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:03.410056 (+ 11263us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:03.410129 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.410212 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.410343 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2311,"prepare.queue_time_us":92954,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":4617,"raft.queue_time_us":2423,"spinlock_wait_cycles":7981824,"thread_start_us":196039,"threads_started":1}
I20260709 13:12:03.382107 32737 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:03.378180 32745 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:03.416352 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.426201 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.428233 29151 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:03.428359 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.429666 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.431558 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.432479 29084 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:03.431876 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.433604 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.436385 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:03.437309 29077 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:03.398566 32693 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.401350 32630 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.440220 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.441854 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.445531 29078 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:12:03.446153 29078 pending_rounds.cc:85] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 5
I20260709 13:12:03.446444 29078 pending_rounds.cc:107] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:12:03.446734 29078 raft_consensus.cc:2889] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:03.449716 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:03.445458 29076 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:03.460702 29076 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:03.429637 32748 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.465173 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:03.466958 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:03.467566 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.426997 32671 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:03.549350 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.531080 29715 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:03.476287 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.542986 29077 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:12:03.497052 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 81045.2us
I20260709 13:12:03.546590 29154 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:03.548570 29083 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:03.543361 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.473454 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:03.549002 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:03.547435 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:03.562041 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica befda4a6c8344c26b0f139e4e1686b3b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:03.543736 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:03.441632 32672 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:03.497306 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.495754 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1576) took 2239 ms. Trace:
I20260709 13:12:03.468576 29076 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:12:03.563896 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 87733.3us
W20260709 13:12:03.539453 32355 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.000s
I20260709 13:12:03.568408 29155 rpcz_store.cc:276] 0709 13:12:01.256162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.567301 (+1311139us) service_pool.cc:224] Handling call
0709 13:12:02.584751 (+ 17450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:03.316307 (+731556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.495503 (+179196us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b5400056254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:03.495687 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616759680}
I20260709 13:12:03.557520 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:03.533576 32681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.008s	sys 0.029s
W20260709 13:12:03.524204 32628 consensus_meta.cc:287] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.005s
I20260709 13:12:03.540833 32546 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:03.574263 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:03.544781 32552 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:03.573397 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:03.543253 32025 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:03.579066 29078 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:03.578487 29008 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:03.575099 32757 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:03.582916 32687 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 1.182s	user 0.000s	sys 0.007s
I20260709 13:12:03.585621 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1441) took 1284 ms. Trace:
I20260709 13:12:03.631685 29079 rpcz_store.cc:276] 0709 13:12:02.300922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.301146 (+   224us) service_pool.cc:224] Handling call
0709 13:12:02.301434 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:03.118475 (+817041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.585376 (+466901us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b5400023d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:03.585556 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877203328}
I20260709 13:12:03.587040 29010 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:03.587875 32724 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.572150 32467 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:03.538113 32692 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.463321 32630 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:03.580446 32681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.008s	sys 0.034s
W20260709 13:12:03.634604 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 62590.7us
I20260709 13:12:03.644290 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:03.525895 32671 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.630899 29710 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:03.591348 32563 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.494773 32748 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.590444 32552 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:03.635886 32757 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.590149 32732 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.631492 32762 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:03.498072 32590 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.525484 32571 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.637662 32737 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:03.575358 32626 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.673149 29081 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:03.707155 32597 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:03.703934 32628 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.640125 32596 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:12:03.713320 29711 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:03.715692 32671 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.669340 32723 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:03.717280 32723 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.717628 32723 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:03.722194 29010 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:03.670893 32745 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:12:03.724647 32745 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:03.724284 32552 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.725014 32745 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:03.726915 29159 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:03.728820 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:03.660522 32025 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:12:03.730842 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:03.747577 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:03.708797 32628 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.667737 32563 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.815970 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:03.817289 29078 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:12:03.819607 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:03.825778 29159 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:03.725386 32552 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:03.731829 32671 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.755497 32737 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.757431 32626 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:03.821295 32568 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:03.806746 32625 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:03.841099 32556 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:03.855192 32568 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.855497 32556 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.837750 32570 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.845050 32752 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:03.848852 32760 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:12:03.848867 32745 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.828629 32624 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:03.873273 32624 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.828925 32565 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:12:03.853189 32737 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.710129 32672 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.860008 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:03.860415   304 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:12:03.755970 32510 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:12:03.858539 29159 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:03.710675 32762 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.870136 32625 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.871533 32752 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.858472 32568 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:03.887291 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:03.889814 29008 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:12:03.892149 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.827502 32604 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:03.905547 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:03.864629 32570 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.950086 29006 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:12:03.896308 32766 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.903424 32355 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.966473 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:03.967321 29156 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:03.967439 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:03.967866 29156 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:03.951129 32625 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:03.970577 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:03.970546 29005 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.976613 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:03.968921 29156 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.956898 32745 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:03.897656 32671 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.991477 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:34936 (request call id 1985) took 3239 ms. Trace:
I20260709 13:12:03.987428 29156 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:03.965368 32692 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.965082 32724 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:03.968698 32766 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.978842 32355 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:03.958664 32604 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:03.956099 32676 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:12:03.994453 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:04.047262 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52898.6us
I20260709 13:12:04.048190 32669 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:04.010862 29005 rpcz_store.cc:276] 0709 13:12:00.753373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.471195 (+717822us) service_pool.cc:224] Handling call
0709 13:12:03.945265 (+2474070us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54001d6fd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115838a3d 00007f411a808cd0 00007f4113d8d254 00007f4113d8d1e2 00007f4113d8d111 00007f4113d8d08c 00007f4113d8c342 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:03.980760 (+ 35495us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465319168}
W20260709 13:12:04.048601 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.996122 32706 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:04.024870 29156 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:04.000258 32611 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:12:04.000458 32559 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:12:04.002943 32732 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.000816 32759 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:12:03.993326 32751 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:04.064364 32759 consensus_queue.cc:1243] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:04.064479 32751 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.065114 32751 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:04.067600 32764 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:12:04.073326 32625 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.075419 32751 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.084425 32556 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:04.084723 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.085446 32556 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:04.085983 32556 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.087642 32630 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.090603 32762 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.091228 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.091320 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.091888 32752 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.092259 29010 raft_consensus.cc:2410] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:12:04.093045 32563 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.094923 32408 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:04.104552 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 321974us
I20260709 13:12:04.093364 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.107802 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.061640 32706 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:04.111416 29156 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:12:04.114347 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.115906 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:04.119014 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:04.119974 29156 raft_consensus.cc:2393] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:04.126065 32619 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:04.062223 32724 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:12:04.132786 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.107267 32737 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:04.128937 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:04.125480 32706 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:04.083892 32723 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.154692 29710 raft_consensus.cc:2410] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:12:04.132272 32619 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.156780 32619 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:04.157274 32626 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:04.158017 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 472) took 1278 ms. Trace:
I20260709 13:12:04.157441 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.158931 29005 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:04.160205 32766 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.158356 29713 rpcz_store.cc:276] 0709 13:12:02.879306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.880592 (+  1286us) service_pool.cc:224] Handling call
0709 13:12:02.917901 (+ 37309us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:03.717418 (+799517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:03.717448 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:03.718070 (+   622us) log.cc:844] Serialized 52 byte log entry
0709 13:12:03.897825 (+179755us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:03.954900 (+ 57075us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.955171 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.086306 (+131135us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b5400576f54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:04.086369 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:04.157739 (+ 71370us) raft_consensus.cc:1685] finished
0709 13:12:04.157828 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:04.157964 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51409,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2595,"raft.queue_time_us":1710,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":33627,"spinlock_wait_cycles":1837707648,"thread_start_us":179402,"threads_started":1,"wal-append.queue_time_us":390291}
I20260709 13:12:04.110363 32751 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.161972 32748 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.162271 32604 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.165102 32736 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:04.165536 32736 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:04.166320 32355 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.112835 32630 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:12:04.173354 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:04.139580 32628 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.176328 32570 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:04.176397 32752 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:04.178001 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.179804 29050 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:04.180970 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:04.146423 32732 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:04.191381 32671 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.149253 32563 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:12:04.194886 32736 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.197504 32736 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:04.201036 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 208407us
I20260709 13:12:04.185627 32762 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:04.201624 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 153340us
I20260709 13:12:04.207187 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:04.209106 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:04.211879 29682 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:04.209739 29079 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:04.195184 32584 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.201251 32671 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.219106 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:04.215384 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.195842 32752 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:04.214607 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:04.218323 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.200919 32669 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:04.221714 32584 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.204984 32762 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:04.234910 32762 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:04.222937 32630 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:04.223109 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.223389 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:04.223354 32671 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.200919 32604 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:12:04.202102 32732 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:04.219729 29078 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:04.230743 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:04.270782 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 47442.3us
I20260709 13:12:04.271023 32630 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.215368 32614 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:12:04.281679 29084 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:04.232985 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:04.272137 29159 raft_consensus.cc:2410] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:12:04.276891 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 46218.9us
I20260709 13:12:04.282768 32630 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:04.238736 32676 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:04.287724 32676 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.289402 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:04.275961 32619 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:04.304538 32619 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.304816 29151 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:04.304991 32619 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:04.305416 32212 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:04.306066 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.306262 32212 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:04.307186 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.307420 32212 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.228818 32552 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:12:04.311673   320 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:04.312162   320 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:04.313093   320 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.232012 32355 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:12:04.315231 29153 pending_rounds.cc:85] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 3
I20260709 13:12:04.319006 29153 pending_rounds.cc:107] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 13:12:04.319466 29153 raft_consensus.cc:2889] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:04.288585 32676 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:04.316622 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:04.315757 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 82947.5us
I20260709 13:12:04.327167 29151 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
W20260709 13:12:04.326473 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.329579 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.331554 29151 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:12:04.330591 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:04.238587 32751 consensus_queue.cc:1059] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:12:04.278991 32672 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:12:04.325060   309 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:12:04.342990 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 122392us
I20260709 13:12:04.281735   319 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:04.353960 29682 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:04.349979 32672 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.370536 32672 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:04.368009 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.371879 32689 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:04.372319 32689 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.372682 32689 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:04.375164 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:04.375743 29004 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:04.365891 32724 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.341302 32563 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:04.385454 32724 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:04.394062   317 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:12:04.385596 32745 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:04.417011 32745 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.419536 32745 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:04.420755 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.365126   319 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:04.407064 32706 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.355280 32564 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.361965 29151 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:04.415594 32671 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.406464 29008 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:04.429709 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.436182 29003 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:04.358879 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:04.341631 32552 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:04.438442 29159 raft_consensus.cc:2410] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:04.432826 29004 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:04.380936 32689 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.435913 32564 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.462769 32564 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:04.463693 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.365990 32669 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.381937 32732 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.429601 32563 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.489705 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.490639 29004 raft_consensus.cc:2410] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:04.490752 32619 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.491648 32735 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:04.492069 32735 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:04.492976 32735 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.494150 32735 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:04.450225 32552 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:04.497510 32552 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.498636 32552 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:04.491227 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.519203 29151 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:04.523233 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.531606 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:04.532735 29155 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:04.537716 29004 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:12:04.538136 29004 pending_rounds.cc:85] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 6
I20260709 13:12:04.538352 29004 pending_rounds.cc:107] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:12:04.538594 29004 raft_consensus.cc:2889] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:04.521798 32570 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:04.589604 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 47242.5us
I20260709 13:12:04.592832 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.593657 29003 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:04.595435 29159 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:12:04.600482 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.533367 32409 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:04.602372 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:04.606055 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61676.2us
I20260709 13:12:04.607005 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.611121 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.613579 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.610582 29717 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
W20260709 13:12:04.608630 32685 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 2.434s	user 0.012s	sys 0.014s
I20260709 13:12:04.601516 32590 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:04.598537 32570 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.643718 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:04.643954 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:04.644567 29081 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:12:04.644645 29083 raft_consensus.cc:2410] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:04.645212 29079 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:04.646354 29084 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:04.610100 32626 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
W20260709 13:12:04.663775 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.610165 32409 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.622808 32732 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.639132 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:04.672485 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:04.673292 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:04.680953 29078 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:04.622619 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.682199 32732 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:04.684691   331 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:12:04.684808   333 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:12:04.687891 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:04.688668 29079 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:04.614643 32630 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.648658 32564 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.684971 32723 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:04.693856 32732 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.695727 32409 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.632277 32696 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.696956 32696 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:04.608151   319 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:04.707206 29079 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:04.700871 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.701090 32736 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:12:04.686512 32724 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.630129 32590 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.698205 32741 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:12:04.714257 32590 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.686925 32619 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:04.716598 32563 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.717077 32630 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:04.681578 32681 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:04.733445 29154 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:04.735011 29078 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
W20260709 13:12:04.695744   328 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:12:04.717941 32681 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:04.761675 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.738070 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 121714us
I20260709 13:12:04.691747 32568 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:04.701799 32706 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.884634 29151 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:04.777274 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.872637 32757 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.892998   320 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:04.849417 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.872592 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.894170 29717 raft_consensus.cc:2368] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:12:04.836710 29051 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:04.895056 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 58467us
I20260709 13:12:04.895694 29051 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:04.894615 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.896394 32749 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:04.897035 32749 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:04.897378 32749 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.897716 32749 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:04.834578 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:04.901126 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.901553 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:04.902199 29004 raft_consensus.cc:2368] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10.
I20260709 13:12:04.848644 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.895907 32681 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:04.908432 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.859439 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.836202 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:04.913056 29010 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:04.835170 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.828670 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:04.916263 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 87694.2us
I20260709 13:12:04.849574 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:04.703922 32564 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:04.829566 32212 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.303s
I20260709 13:12:05.080706 32315 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:05.003471 29081 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:05.131561 32666 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:05.156744 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 135626us
I20260709 13:12:05.009315 29083 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:05.011041 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:04.888274 32762 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.653s	user 0.000s	sys 0.012s
I20260709 13:12:05.013068 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:04.937067 29051 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:04.911336 29001 raft_consensus.cc:2376] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9. Re-sending same reply.
W20260709 13:12:05.161108 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 224187us
W20260709 13:12:05.161765 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.156208 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.162578 29076 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:04.920017 32568 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:12:04.965647   345 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.091s	user 0.000s	sys 0.005s
I20260709 13:12:05.164667 32568 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:05.164808 32564 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.150097 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.042347 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52580 (request call id 17) took 3533 ms. Trace:
I20260709 13:12:05.165819 29081 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
W20260709 13:12:04.872462   337 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:12:05.163998 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.157109 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.166005 29150 rpcz_store.cc:276] 0709 13:12:01.486992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.207131 (+720139us) service_pool.cc:224] Handling call
0709 13:12:05.020578 (+2813447us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54002a3ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583b853 00007f411a80960a 00007f4113d8f7d4 00007f4113d8f762 00007f4113d8f691 00007f4113d8f60c 00007f4113d8e8c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:05.020677 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681762688}
W20260709 13:12:05.129384 32764 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:12:04.771890 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:04.813520 32570 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:04.737957 32689 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:04.965845   344 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:12:04.992368   347 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
I20260709 13:12:05.077356 32571 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:04.966030 32745 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:05.149255 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 301406us
I20260709 13:12:05.205665 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:05.202775 32571 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:05.207132 32571 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.190279 32564 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:05.208350 32571 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:05.204699 32745 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:05.211104 32745 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:05.211351 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.194377 32666 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.213099 32666 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.215026 32666 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.188788   340 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.216732 29051 leader_election.cc:400] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:12:05.216552   340 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.217056 29051 leader_election.cc:403] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:05.217877 32727 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.218621   340 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:04.808987 32599 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.193411 32568 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.197196 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:04.992486 32672 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.168428 32757 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.169064 32754 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:04.768323 32676 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:05.187326 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:05.220615 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 59636us
I20260709 13:12:05.173077 32696 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.229419 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.253014 29082 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:05.224314 29122 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.254230 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.195683   346 consensus_queue.cc:1059] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:12:05.254928 32696 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.255424 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.255699   346 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:05.253159 32666 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.256417   346 raft_consensus.cc:3055] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:05.071599 32664 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.256543 32666 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.256814 29713 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:12:05.257714 32664 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:05.228163 32550 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:12:05.218356 32727 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.244750 32625 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.200183 32712 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.231017 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.229372 32706 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.237491 32568 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:05.259428 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.256798   346 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:05.245389 32749 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.297681 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:05.256548 32676 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.169260 32762 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.267329 29002 raft_consensus.cc:2410] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:05.270370 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.298204 32552 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.265228 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:05.314739 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 49591.7us
I20260709 13:12:05.315900 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.317366 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.272094 29082 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:05.272895 32758 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:05.271795 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.319092 29001 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:05.270532 32712 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.273372 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.248860 32672 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.273910 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:05.273838 32560 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.274215 29009 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:05.275559 29006 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:12:05.276176 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.278267 32667 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:05.278704 32751 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:05.263128 32566 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.283761 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1827) took 1822 ms. Trace:
I20260709 13:12:05.284590 32666 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.265501   319 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:05.261569 32665 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:05.287272 32706 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.288132 32574 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:05.352020 32665 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.290493 32727 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:05.293043   341 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:05.295770 32705 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:05.251593 32757 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.354740 32665 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.352710 32574 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:05.356248 32757 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.276201 32603 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.356212 32574 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.305320 29083 raft_consensus.cc:2393] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:05.309273 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:05.271018 29078 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:05.325691   346 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.356947 32624 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.348150 32566 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.300491 32766 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.360276 32751 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.360484 32712 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:05.360805 32751 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.363431 32751 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.302112   320 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.338104 32597 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:12:05.364318   320 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.364954 32597 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:05.365300 32597 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:05.301853 32724 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:05.366072 32597 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.366643 32724 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.367210 32597 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:05.368280 32749 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.369288 32724 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.325237 32552 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:05.352665   351 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:12:05.338542 32315 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.372385 32315 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.360291 32766 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.354764 32705 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.374565 32766 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.375674 32705 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.378108 32745 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.381039 32596 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.381556 32354 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:05.381507 32596 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.383654 32596 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:05.384191 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:05.384974 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.385046 32706 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:05.385674   320 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.387725 32696 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.388275 32696 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.388734 32696 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.390676 32696 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.391156 32696 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:05.359845 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.392894 29078 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:05.393285 29080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1449) took 2236 ms (client timeout 2043 ms). Trace:
W20260709 13:12:05.393666 29080 rpcz_store.cc:269] 0709 13:12:03.156729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.157571 (+   842us) service_pool.cc:224] Handling call
0709 13:12:05.393047 (+2235476us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001211d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:05.393214 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2433549184}
I20260709 13:12:05.394742 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.395231 29080 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:05.398428 29002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2301) took 2552 ms (client timeout 2516 ms). Trace:
I20260709 13:12:05.358541   346 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:05.379882 32765 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 1.443s	user 0.001s	sys 0.001s
I20260709 13:12:05.401758 29716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 470) took 2523 ms. Trace:
I20260709 13:12:05.402087 29716 rpcz_store.cc:276] 0709 13:12:02.878690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.880414 (+  1724us) service_pool.cc:224] Handling call
0709 13:12:02.906300 (+ 25886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.906747 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.906787 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.937296 (+ 30509us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b2c0004cb58. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:12:02.937691 (+   395us) log.cc:844] Serialized 26 byte log entry
0709 13:12:03.418474 (+480783us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b18003c00a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:12:03.465538 (+ 47064us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:03.465587 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.465655 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:05.379354 (+1913699us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:05.401539 (+ 22185us) raft_consensus.cc:1685] finished
0709 13:12:05.401561 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.401678 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22966,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":4662,"spinlock_wait_cycles":57289600,"thread_start_us":46874,"threads_started":1,"wal-append.queue_time_us":903872}
I20260709 13:12:05.403156 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.371765 32552 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.333876 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.404731 29082 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:05.405014 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.405282 32603 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:05.405822 32762 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.406165 32603 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:05.406690 32603 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:05.342382 29077 rpcz_store.cc:276] 0709 13:12:03.461409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.461532 (+   123us) service_pool.cc:224] Handling call
0709 13:12:03.542762 (+ 81230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:03.561093 (+ 18331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.283558 (+1722465us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b5400337050. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:05.283703 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185909504}
I20260709 13:12:05.408303 32315 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.318390 32546 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:05.409355 32546 raft_consensus.cc:697] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:05.410199 32546 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.325548 32735 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:05.331277 32467 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:05.412524 32735 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:05.334955 32564 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:05.412642 32596 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:05.413043 32735 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.413196 32564 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:05.413894 32564 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:05.398828 29002 rpcz_store.cc:269] 0709 13:12:02.845932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.846119 (+   187us) service_pool.cc:224] Handling call
0709 13:12:05.398206 (+2552087us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54003aaa54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:05.398329 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5483959168}
I20260709 13:12:05.349228   319 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.419888 32754 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.357483 32574 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:05.405640 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.422204 32754 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.423177 29082 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:05.313684 32669 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.424541 32754 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.424428 32669 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.427064 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:05.427955 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:05.428915 32736 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:05.429467 32736 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:05.430250 32736 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.432462 32467 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:05.432981 32467 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:05.433774 32467 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.437717 32712 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:05.439386   348 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.132s	user 0.002s	sys 0.000s
I20260709 13:12:05.441829 32672 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:05.443702 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58804.4us
W20260709 13:12:05.444190 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 135023us
I20260709 13:12:05.446425 29009 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:05.446745 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:05.447261 29083 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:05.447932 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:05.448427 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:05.449039 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.449851 29082 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:05.449110 29712 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:05.451797 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:05.403728 29081 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:05.455488 32025 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.457237 32722 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.457899 32722 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.458623 32665 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.456001 32025 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.461313 29079 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:05.461921 32025 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.451860 29082 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:05.467029 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:05.467556 32584 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.467052 32722 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:05.469081   319 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.332450 29122 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:05.471042 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 86060.3us
I20260709 13:12:05.471755 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:05.473820 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:05.474310 29007 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:05.474483 29153 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:05.477640 29150 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:05.481413 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.481999 29005 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
W20260709 13:12:05.484432 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica befda4a6c8344c26b0f139e4e1686b3b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:05.484750 29080 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:05.485893 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.487447 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.488018 29005 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:12:05.488386 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.490031 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.489533 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:05.490185 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 157836us
I20260709 13:12:05.491313 29122 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.494282 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.494947 29005 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:05.495399 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.486513 29006 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:05.496157 29003 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:05.442575 32712 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.500550 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.496397 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.501380 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.505288 29085 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:05.505539 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.506121 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.498200 29006 pending_rounds.cc:85] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:12:05.506695 29085 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:05.506898 29006 pending_rounds.cc:107] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:12:05.507457 29006 raft_consensus.cc:2889] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:05.507917 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.499848 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.500005 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.509195 29081 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:05.509536 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.499365 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.497223 29712 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:05.496279 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.499297 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.501889 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.497809 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.511270 29079 raft_consensus.cc:2410] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:05.512384 29051 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:05.486008 29078 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:05.512496 29083 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:12:05.513068 29083 pending_rounds.cc:85] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 4
I20260709 13:12:05.513437 29083 pending_rounds.cc:107] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:12:05.506367 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.514258 29083 raft_consensus.cc:2889] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:05.510124 29151 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:05.459105 32665 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.495478 32748 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.515974 29153 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:12:05.532718 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.517347 29005 raft_consensus.cc:2368] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:12:05.548862 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.518486 29002 raft_consensus.cc:2410] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:12:05.549759 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.522782 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.523222 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.525835 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.551776 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.552094 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.553571 29083 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:05.518714 29122 leader_election.cc:400] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10.
I20260709 13:12:05.529991 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.554066 29122 leader_election.cc:403] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:12:05.554786 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45409.3us
I20260709 13:12:05.554972 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.526897 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.555498 29122 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.529665 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.516845 29080 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:12:05.556874 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.559005 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.560429 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.561899 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.563243 29153 raft_consensus.cc:2368] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
W20260709 13:12:05.572394 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.498339 32025 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:05.501564 32681 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:05.593268 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.594421 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.578649 29085 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:05.581887 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.584197 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.502841 32672 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:05.596457 29004 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:05.587368 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.600488 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.601511 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.602248 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.603643 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.606914 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.527352 32696 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:05.517421 32570 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:05.611841 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.615041 29158 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:05.606560 29159 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
W20260709 13:12:05.620735 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 61798.2us
I20260709 13:12:05.624718 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.632390 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:05.633102 29155 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:05.638316 32550 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:12:05.585184 32727 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:05.584977 32630 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:05.640620 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.584847 32748 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.641690 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.643404 32665 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.643573 32748 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.644848 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.610059 32669 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.646287 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.647504 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.582402 32568 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10.
I20260709 13:12:05.597028 32745 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.575537 32566 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.667434 29005 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:05.668838 29005 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:05.672719 29076 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:12:05.660584 29004 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:05.648021 29153 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:12:05.656596 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:05.661417 29050 leader_election.cc:400] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 6. Current term is 7.
I20260709 13:12:05.675266 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.675490 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.676553 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.663806 28975 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.672824 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.656965 29007 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:05.677290 29153 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:05.677964 29005 raft_consensus.cc:2410] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 13:12:05.674252 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.678283 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.678678 29004 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:05.679394 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.679831 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.680027 29005 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:05.680429 29004 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:12:05.681250 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:05.592823 32681 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:05.684809 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:05.693693 29712 raft_consensus.cc:2410] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:05.605235 32722 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.686335 28975 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:05.695209 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.684626 32025 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:05.652248 32737 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:05.709002 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:05.672504   321 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.004s	sys 0.041s
W20260709 13:12:05.713032   321 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.066s	user 0.004s	sys 0.042s
I20260709 13:12:05.611042 32672 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.716584 32672 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:05.692945 32748 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:05.695865 32681 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:05.644845 32696 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:05.721076 32696 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.649049 32712 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:05.693887   346 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:12:05.679630 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.728503 32614 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:05.731123 29155 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:05.732138 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:05.732820 29155 raft_consensus.cc:2368] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:12:05.735406 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.739475 32735 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.742283 32735 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.762321 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:05.762961 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52585.1us
I20260709 13:12:05.763823 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:05.764946 29682 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:05.692207 32566 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:05.770533 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:05.773317 29122 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:05.705238 32437 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:05.777895 32570 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:05.776813   346 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:05.707217 32722 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.785564 29156 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:05.785224 32570 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.788163 32570 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:05.749648 32556 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.795303 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.796795 32570 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.749799 32712 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:05.797799 32712 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:05.798929 32570 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:05.778729 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.778892 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.800742 29157 raft_consensus.cc:2393] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:05.801586 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.781291 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:05.780025 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:05.802556 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8340da85294490fb88f62f80393531f"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:05.804119 29159 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:05.806054 29159 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:05.810554 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.812498 29085 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:05.811403 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:05.762602 32571 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:05.762434 32745 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:05.810979 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:05.821799 29158 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:05.823036 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:05.776655 32212 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:05.839154 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.732340 32737 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:05.732457 32628 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:12:05.786926   357 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:05.786701 32568 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:05.786839 32735 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:05.847107 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.837973   346 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:05.844131 29683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.028s	sys 0.062s
I20260709 13:12:05.837646 32571 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:05.902303 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 55304.1us
I20260709 13:12:05.911698 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.847440 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:12:05.902979 32666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.006s	sys 0.032s
W20260709 13:12:05.904596 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.776408 32564 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:05.908564 29683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.028s	sys 0.062s
I20260709 13:12:05.846771 32212 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.845314 32737 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:05.848799 32025 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:05.920931 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.902030 32568 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:05.909789 32603 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.003s
I20260709 13:12:05.934361 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:05.918231 32666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.006s	sys 0.033s
I20260709 13:12:05.917176 32571 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:05.939769 29008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.013s	sys 0.021s
I20260709 13:12:05.908118 32712 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:05.955039 29008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.013s	sys 0.022s
I20260709 13:12:05.946295 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:05.904178   357 raft_consensus.cc:697] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:05.846472 32745 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.937980 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:05.974799 29009 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:05.935352 32727 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:05.958925 29158 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:05.953866 32025 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:05.916796 32630 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:05.932030 32212 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:05.957192 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:05.937669 32568 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:05.956028 32597 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:05.917454   346 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:05.978241 32712 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:05.901686 32735 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:05.934151 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 645411us
I20260709 13:12:05.944420 32354 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:12:05.955410 32603 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:06.009603 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52498.3us
I20260709 13:12:06.126308 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.127405 29077 raft_consensus.cc:2410] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:12:06.132440 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.135352 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:06.037508 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:06.148074 29006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.020s	sys 0.082s
W20260709 13:12:06.148684 29006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.021s	sys 0.082s
I20260709 13:12:06.151589 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.159193 29152 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:06.160202 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:06.162704 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.161934 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:06.027789 32735 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:06.166738 29159 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:06.031044 32718 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.146s
I20260709 13:12:06.033450 32574 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:06.120601 29124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.000s	sys 0.008s
I20260709 13:12:06.190310 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:06.226811 29124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.006s	sys 0.010s
I20260709 13:12:06.177860 29001 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:06.028838   363 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.159s
I20260709 13:12:06.242489 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:06.184567 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.247454 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:06.041654 29085 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:06.177187 32757 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:06.072789   346 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:06.055644 29003 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:06.063468 29712 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:06.034004 32722 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:05.998541 29009 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:06.062952 32745 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:06.270848 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.194275 29150 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:06.344998 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.345615 29085 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
W20260709 13:12:06.119777 29084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.002s	sys 0.006s
I20260709 13:12:06.346494 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.180732 29152 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 13:12:06.346906 29084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.002s	sys 0.013s
W20260709 13:12:06.178893 32736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.044s	sys 0.041s
I20260709 13:12:06.245596 29717 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:06.245019 29710 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:06.244349 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:06.180148 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.354552 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:06.289424 29004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.014s	sys 0.017s
W20260709 13:12:06.355262 29004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.014s	sys 0.017s
I20260709 13:12:06.289786 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.356639 29079 raft_consensus.cc:2393] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:06.229288   364 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:12:06.322965 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.194416 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.366726 29077 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:06.367841 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.368582 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.369005 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:06.198160 29005 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
W20260709 13:12:06.228745   368 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 13:12:06.241865 32681 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.523s	user 0.005s	sys 0.006s
W20260709 13:12:06.120703 32552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.008s	sys 0.002s
I20260709 13:12:06.288676 32467 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.364203 29150 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:06.365079 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.366770 29082 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
W20260709 13:12:06.120646 29048 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.003s	sys 0.006s
W20260709 13:12:06.407271 29048 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.003s	sys 0.006s
W20260709 13:12:06.407673 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 845850us
W20260709 13:12:06.361894 32736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.044s	sys 0.050s
I20260709 13:12:06.371408 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.371779 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:06.325723   360 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 13:12:06.373633 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.348304 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:06.377895 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:06.226118 32667 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.225927 32666 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.196734 32354 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:06.363828 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.387380 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.361804 32566 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.342905 32757 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:06.238636 32568 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.388233 32681 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.053386   357 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:06.388928 32552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.009s	sys 0.008s
W20260709 13:12:06.399935   367 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 13:12:06.324054 32727 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:06.350715 32603 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:06.096684 32564 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:06.414148 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1450) took 1216 ms. Trace:
I20260709 13:12:06.532964 29153 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:06.415766 32212 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:06.424718 29123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.028s	sys 0.034s
W20260709 13:12:06.536357 29123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.029s	sys 0.034s
I20260709 13:12:06.450466   376 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:06.533461 29084 rpcz_store.cc:276] 0709 13:12:05.197706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.407231 (+209525us) service_pool.cc:224] Handling call
0709 13:12:05.462687 (+ 55456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:05.490301 (+ 27614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.490332 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:05.490820 (+   488us) log.cc:844] Serialized 52 byte log entry
0709 13:12:06.347311 (+856491us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:06.347349 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.347407 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.413875 (+ 66468us) raft_consensus.cc:1685] finished
0709 13:12:06.413901 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.414073 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27236,"raft.queue_time_us":30002,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":798914,"threads_started":1,"wal-append.queue_time_us":906327}
I20260709 13:12:06.538508 28975 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:06.427697 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:06.351222 32671 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.436583 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 609) took 1303 ms. Trace:
I20260709 13:12:06.438206 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:06.437980 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 89781.1us
I20260709 13:12:06.545249 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:06.439782 29081 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:06.546646 29081 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:06.246961 32737 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.447691 29077 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:06.451673 29085 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:06.487732 29082 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:06.436203 32748 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.550956 32748 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:06.485546 32571 consensus_meta.cc:287] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.508s	user 0.005s	sys 0.005s
I20260709 13:12:06.446166 32692 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.552331 32692 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:06.553279 32748 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:06.553818 32748 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.554246 32748 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:06.555068 32692 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:06.542115 29076 rpcz_store.cc:276] 0709 13:12:05.133270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.196958 (+ 63688us) service_pool.cc:224] Handling call
0709 13:12:06.436449 (+1239491us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54000244d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:06.436537 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981771136}
I20260709 13:12:06.541211 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:06.561625 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.562582 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2042) took 1289 ms. Trace:
I20260709 13:12:06.547595 29081 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.563884 29081 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:06.563071 29010 rpcz_store.cc:276] 0709 13:12:05.272838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.276628 (+  3790us) service_pool.cc:224] Handling call
0709 13:12:05.276928 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:05.277165 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.277194 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:05.277659 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:12:05.657337 (+379678us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:05.657376 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.657439 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.562335 (+904896us) raft_consensus.cc:1685] finished
0709 13:12:06.562369 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.562514 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":379302,"threads_started":1,"wal-append.queue_time_us":693146}
I20260709 13:12:06.565068 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:06.444721 32666 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.570039 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:06.570960 29004 raft_consensus.cc:2368] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:12:06.444159 32667 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.571985 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:06.450251 32566 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.572369 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.432873 32467 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.561177   320 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:06.556362 32571 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.558344 32354 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.579569 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.560699 32671 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.589110 29081 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:12:06.589556 29081 pending_rounds.cc:85] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 6
I20260709 13:12:06.589859 29081 pending_rounds.cc:107] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:12:06.590129 29081 raft_consensus.cc:2889] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:06.533775 32727 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:06.587399 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 568) took 1201 ms. Trace:
I20260709 13:12:06.566323 32603 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:12:06.580521   320 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.568434   376 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:06.608095 32737 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:06.610953 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2311) took 1123 ms. Trace:
I20260709 13:12:06.611339 29001 rpcz_store.cc:276] 0709 13:12:05.487387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.488182 (+   795us) service_pool.cc:224] Handling call
0709 13:12:06.610774 (+1122592us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b540032cdd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:06.610898 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1583728000}
I20260709 13:12:06.583196 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.596451 29005 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:06.601382 29008 rpcz_store.cc:276] 0709 13:12:05.386222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.425555 (+ 39333us) service_pool.cc:224] Handling call
0709 13:12:05.425887 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:05.426268 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.426305 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:05.426788 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:12:05.955845 (+529057us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:05.955900 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.955993 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.587158 (+631165us) raft_consensus.cc:1685] finished
0709 13:12:06.587184 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.587334 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4054,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":528599,"threads_started":1,"wal-append.queue_time_us":712833}
I20260709 13:12:06.612888 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.613649 29001 raft_consensus.cc:2410] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:12:06.613725 29008 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:06.574955 32705 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:06.616344 29010 raft_consensus.cc:2410] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:12:06.608238   376 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:06.618932 29154 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.019s	sys 0.055s
W20260709 13:12:06.619326 29154 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.019s	sys 0.055s
I20260709 13:12:06.621042 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:06.586723 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:06.624555 29004 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 6.
I20260709 13:12:06.625291 32566 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:06.603837   320 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:06.589268 32354 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.625303 32603 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:06.617606 32748 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:06.671083 29007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.007s	sys 0.016s
W20260709 13:12:06.671701 29007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.007s	sys 0.016s
I20260709 13:12:06.651996 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1622) took 1120 ms. Trace:
I20260709 13:12:06.683423 29155 rpcz_store.cc:276] 0709 13:12:05.531827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.810207 (+278380us) service_pool.cc:224] Handling call
0709 13:12:05.810505 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.810854 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:06.151996 (+341142us) log.cc:844] Serialized 14 byte log entry
0709 13:12:06.651671 (+499675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:06.651726 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:06.651766 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.651828 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.651941 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":600884,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":101,"thread_start_us":805941,"threads_started":2}
I20260709 13:12:06.686098 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:06.619614 32705 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:06.651897 32603 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:06.654105 29083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.024s	sys 0.021s
W20260709 13:12:06.727780 29083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.025s	sys 0.021s
W20260709 13:12:06.709939 29151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.020s	sys 0.029s
I20260709 13:12:06.711074 29155 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:06.673559 29156 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.013s	sys 0.044s
W20260709 13:12:06.745765 29156 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.015s	sys 0.046s
W20260709 13:12:06.753618   380 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 13:12:06.750065   379 log.cc:927] Time spent T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:12:06.759333 32354 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:06.729010 32681 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:06.727219 32568 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:06.729663 32603 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:06.792438 32667 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:06.795133 32603 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.797736 32671 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:06.810264 32723 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.019s	sys 0.008s
I20260709 13:12:06.909770 32571 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:06.910501 32723 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.020s	sys 0.008s
W20260709 13:12:06.869971 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.912153 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.796891 32467 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:06.912716 32705 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:06.870745 28976 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.008s	sys 0.008s
W20260709 13:12:06.913630 28976 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.008s	sys 0.008s
W20260709 13:12:06.914251 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 646208us
I20260709 13:12:06.885780 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2053) took 1148 ms. Trace:
I20260709 13:12:06.916414 29007 rpcz_store.cc:276] 0709 13:12:05.737425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.796337 (+ 58912us) service_pool.cc:224] Handling call
0709 13:12:05.796681 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:05.797008 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.797037 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:05.797463 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:12:06.672231 (+874768us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:06.672290 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.672381 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.884951 (+212570us) raft_consensus.cc:1685] finished
0709 13:12:06.885041 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.885525 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19284,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2570,"thread_start_us":874307,"threads_started":1,"wal-append.queue_time_us":981056}
I20260709 13:12:06.887337 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:06.888741 29078 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.010s	sys 0.005s
W20260709 13:12:06.924292 29078 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.010s	sys 0.005s
I20260709 13:12:06.915812 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:06.927866 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:06.929257 32552 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:06.889250 29080 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.020s	sys 0.010s
I20260709 13:12:06.930114 32552 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:06.930450 29080 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.020s	sys 0.010s
I20260709 13:12:06.930984 32552 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:06.932255 32671 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:06.932717 32552 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:06.934679 32671 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:06.935086 32671 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.935434 32671 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:06.890306 29124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.009s	sys 0.008s
W20260709 13:12:06.936687 29124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.009s	sys 0.008s
I20260709 13:12:06.923540 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:06.944598 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:06.745765 29151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.021s	sys 0.030s
I20260709 13:12:06.747030 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:06.899068 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 331924us
W20260709 13:12:06.900629 29683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.003s	sys 0.017s
W20260709 13:12:06.904125 29680 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.007s	sys 0.012s
W20260709 13:12:06.905932 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.766783 32212 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:06.947455 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:06.808619 32727 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:06.911964 32603 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:06.801687 32735 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
I20260709 13:12:06.949627 32736 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:06.896745 29150 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:12:06.953910 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1621) took 1422 ms. Trace:
I20260709 13:12:06.965570 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2332) took 1419 ms. Trace:
I20260709 13:12:06.973694 32571 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:06.974481 32705 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:07.011072 29680 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.007s	sys 0.012s
W20260709 13:12:07.011415 29683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.003s	sys 0.018s
I20260709 13:12:07.013796 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.034894 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2330) took 1489 ms. Trace:
I20260709 13:12:07.039117 32597 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:07.021899 29001 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:07.047072 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.048197 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.048663 32748 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:12:07.049381   407 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:12:07.057866 32025 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:07.050194 32571 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.059490 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2333) took 1512 ms. Trace:
W20260709 13:12:07.059876 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 518559us
I20260709 13:12:07.060798 32571 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.051860 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.062088 29155 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:07.060333 29154 rpcz_store.cc:276] 0709 13:12:05.546264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.066895 (+520631us) service_pool.cc:224] Handling call
0709 13:12:06.067255 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.067498 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:06.620347 (+552849us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.057121 (+436774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.057248 (+   127us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:07.057429 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.058743 (+  1314us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.059047 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32265,"prepare.queue_time_us":344706,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":49345,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":158,"thread_start_us":939913,"threads_started":2}
I20260709 13:12:07.063364 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.057031 29710 raft_consensus.cc:2410] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:12:07.058145 32737 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:07.087905 29002 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:07.088001 32737 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.089030 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 614) took 1050 ms. Trace:
I20260709 13:12:07.089076 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.090883 32737 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:07.089704 29083 rpcz_store.cc:276] 0709 13:12:06.038402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.038563 (+   161us) service_pool.cc:224] Handling call
0709 13:12:06.038940 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:06.039199 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.039237 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:06.039756 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:12:06.728398 (+688642us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:07.012633 (+284235us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.088331 (+ 75698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.088453 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.088533 (+    80us) raft_consensus.cc:1685] finished
0709 13:12:07.088549 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.088766 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32634,"prepare.queue_time_us":94039,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"thread_start_us":970239,"threads_started":2,"wal-append.queue_time_us":772056}
I20260709 13:12:07.065196 29155 raft_consensus.cc:2410] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:12:07.067481 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.067961 32705 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.069212 32735 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.064898   402 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:07.074270 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 127040us
I20260709 13:12:07.076529 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.044420 29159 rpcz_store.cc:276] 0709 13:12:05.531622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.809360 (+277738us) service_pool.cc:224] Handling call
0709 13:12:05.809603 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.225890 (+416287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.953341 (+727451us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400055d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:06.953728 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":820830464}
I20260709 13:12:07.046818 29158 rpcz_store.cc:276] 0709 13:12:05.545377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.842774 (+297397us) service_pool.cc:224] Handling call
0709 13:12:05.850349 (+  7575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.980426 (+130077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.034188 (+1053762us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540013c254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:07.034651 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249153280}
I20260709 13:12:07.084204 32597 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:07.086244 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:07.087319   392 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:12:07.090777 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:07.066817 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1609) took 1539 ms. Trace:
I20260709 13:12:07.074210 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.092305 29002 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:07.110087 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.044767 29156 rpcz_store.cc:276] 0709 13:12:05.545982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.852986 (+307004us) service_pool.cc:224] Handling call
0709 13:12:05.853293 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.026984 (+173691us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:06.747621 (+720637us) log.cc:844] Serialized 14 byte log entry
0709 13:12:06.899599 (+151978us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:06.899728 (+   129us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:06.899890 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.965083 (+ 65193us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54002a2bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:06.965137 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.965380 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":385017,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":14901,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"spinlock_wait_cycles":398898560,"thread_start_us":871512,"threads_started":2,"wal-append.queue_time_us":76735}
I20260709 13:12:07.116174 29158 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:07.117641 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.120209 29157 raft_consensus.cc:606] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:07.120702 29157 raft_consensus.cc:614] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:07.121747 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.122066   402 consensus_queue.cc:1059] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:07.123236   402 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:07.123637   402 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:07.124317 32570 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.124720 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:07.125329 29717 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:07.125239 32570 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.126708 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.127115 29150 raft_consensus.cc:2410] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:12:07.124418 29010 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:07.113292 29715 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
W20260709 13:12:07.130810 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 186455us
I20260709 13:12:07.124377 32667 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:07.133173 32667 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:07.133651 32696 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:07.134292 32667 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.127911 32570 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:07.137802 32735 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:07.128836 29007 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:07.138489 29007 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:07.121766 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:07.140235   399 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:07.140792   399 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:07.140905 29050 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.141324   399 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:07.143136   370 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.564s	user 0.004s	sys 0.004s
I20260709 13:12:07.143242 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:07.143381   400 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:07.144068 29156 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:07.144979 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.146694 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.112875 32737 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:07.148312   400 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:07.148505 32737 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.149827 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:07.150694 29003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33750 (request call id 574) took 1651 ms (client timeout 1730 ms). Trace:
W20260709 13:12:07.151181 29003 rpcz_store.cc:269] 0709 13:12:05.498909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.499098 (+   189us) service_pool.cc:224] Handling call
0709 13:12:07.150303 (+1651205us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54001caf54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:07.150492 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252756864}
I20260709 13:12:07.152185   400 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:07.152395   321 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:07.153033 29007 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.153754 32737 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:07.154253 29007 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:07.154415   321 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:07.154989   321 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:07.147758 32692 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:07.144979 29050 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:07.159310 29153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2346) took 1510 ms (client timeout 1875 ms). Trace:
W20260709 13:12:07.163049 29153 rpcz_store.cc:269] 0709 13:12:05.649083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.679364 (+ 30281us) service_pool.cc:224] Handling call
0709 13:12:07.158890 (+1479526us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54000951d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:07.159070 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849581824}
I20260709 13:12:07.164633 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.164791 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.163205 32666 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.165334 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.165797   400 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:07.166622   400 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:07.167269 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:07.167725 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.167852   400 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:07.168741 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 76761.4us
I20260709 13:12:07.169086   322 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:07.170169 32689 consensus_queue.cc:1059] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:07.165179 32692 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:07.171388 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.171995 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.124001   402 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:07.173322 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1868) took 1044 ms. Trace:
I20260709 13:12:07.172281 29005 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:07.098296 29151 rpcz_store.cc:276] 0709 13:12:05.527283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.527388 (+   105us) service_pool.cc:224] Handling call
0709 13:12:05.527677 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.527861 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:06.052471 (+524610us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.066025 (+1013554us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.066149 (+   124us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:07.066255 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.066399 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.066647 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":842754,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3261,"raft.queue_time_us":2310,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":147,"thread_start_us":1416163,"threads_started":2,"wal-append.queue_time_us":719462}
I20260709 13:12:07.173247 32757 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.165364 29153 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:07.175467 32025 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.175992 32736 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:07.135943 32667 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:07.192790 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.180510 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.161705 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.183902 32727 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:07.151515 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.191332 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:07.171437   322 raft_consensus.cc:3055] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:07.192721 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:07.179059 32735 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:07.196384 29010 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:07.197844 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.197576 32025 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.198596 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.196755   322 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:07.202250 32666 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:07.203244 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:07.203146 32571 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:07.193218 32736 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:07.204411   395 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:07.205148 32603 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:07.205088 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43626.4us
I20260709 13:12:07.205894 29155 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:07.200313 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.231813 29683 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:07.206408   320 consensus_meta.cc:287] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.562s	user 0.005s	sys 0.014s
I20260709 13:12:07.207398 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2314) took 1710 ms. Trace:
I20260709 13:12:07.208637 29050 leader_election.cc:400] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:12:07.233525 29050 leader_election.cc:403] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:12:07.212447 32757 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:07.229288 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:07.235777 29153 raft_consensus.cc:2368] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8.
I20260709 13:12:07.215865 29156 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:07.219488 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.186327 29080 rpcz_store.cc:276] 0709 13:12:06.128511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.128687 (+   176us) service_pool.cc:224] Handling call
0709 13:12:06.129037 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.129291 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:06.931162 (+801871us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.172605 (+241443us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.172834 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.172898 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:07.172969 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.173078 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.173255 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49256,"prepare.queue_time_us":749858,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":87,"raft.queue_time_us":106997,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":299,"thread_start_us":858733,"threads_started":2}
I20260709 13:12:07.131610 32671 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.237344 29151 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:07.236408 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.253608 32748 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:07.262018   399 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.235149 32597 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.264622 32735 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.265494 32671 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.266585 32597 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:07.266749 32735 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:07.232636 29006 rpcz_store.cc:276] 0709 13:12:05.497051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.497743 (+   692us) service_pool.cc:224] Handling call
0709 13:12:05.498026 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:05.508189 (+ 10163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.508233 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:05.508609 (+   376us) log.cc:844] Serialized 26 byte log entry
0709 13:12:06.149336 (+640727us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:06.149392 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.149466 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.142235 (+992769us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:07.206932 (+ 64697us) raft_consensus.cc:1685] finished
0709 13:12:07.206960 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.207230 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6156544,"thread_start_us":640401,"threads_started":1,"wal-append.queue_time_us":1024834}
I20260709 13:12:07.273368 32025 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:07.278807   399 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.281566 29007 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:12:07.282085 29712 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:12:07.283509 29007 pending_rounds.cc:85] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:12:07.284196 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52580 (request call id 18) took 1621 ms. Trace:
I20260709 13:12:07.284659 29157 rpcz_store.cc:276] 0709 13:12:05.663144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.802424 (+139280us) service_pool.cc:224] Handling call
0709 13:12:07.283955 (+1481531us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54002b66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583b853 00007f411a80960a 00007f4113d8f7d4 00007f4113d8f762 00007f4113d8f691 00007f4113d8f60c 00007f4113d8e8c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:07.284095 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3016643456}
I20260709 13:12:07.284253 29007 pending_rounds.cc:107] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 9.6
W20260709 13:12:07.286998   382 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.663s	user 0.001s	sys 0.000s
I20260709 13:12:07.289021 29007 raft_consensus.cc:2889] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:07.292819   363 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:07.293229 29001 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:07.296522 32564 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:07.297104 32564 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:07.298151 32564 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.301992 32354 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.302763 32354 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.303139 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:07.304471 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 153112us
I20260709 13:12:07.305192 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 579) took 1150 ms. Trace:
I20260709 13:12:07.305377 32603 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:07.305616 29008 rpcz_store.cc:276] 0709 13:12:06.154981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.615536 (+460555us) service_pool.cc:224] Handling call
0709 13:12:06.623103 (+  7567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.623296 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.623335 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:06.623380 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:06.623408 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.623459 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.304836 (+681377us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b540032cb50. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:07.304990 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16705280}
I20260709 13:12:07.306473 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.306639 32603 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.308276 32671 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:07.309002 32671 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:07.308542   322 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:07.309823   377 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.700s	user 0.003s	sys 0.000s
I20260709 13:12:07.254282   320 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.316783   320 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.317080 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:07.317487   320 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:07.317919 29159 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:07.318069   320 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.318933 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.320473 29157 raft_consensus.cc:606] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:07.320212   321 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.320968 29157 raft_consensus.cc:614] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:07.321784   321 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:07.311017 29680 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.323412 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:07.315850 29004 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:07.325661 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2339) took 1085 ms. Trace:
I20260709 13:12:07.326063 29004 rpcz_store.cc:276] 0709 13:12:06.239813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.626064 (+386251us) service_pool.cc:224] Handling call
0709 13:12:06.626318 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.324193 (+697875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.325391 (+  1198us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b54003aaa54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:07.325599 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1584884096}
W20260709 13:12:07.315833   414 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
W20260709 13:12:07.313288 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 115749us
I20260709 13:12:07.313524 32705 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.313830 32025 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:07.320575 29150 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:07.312825 32354 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:07.310520 32671 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:07.328492 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 120138us
W20260709 13:12:07.292882 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 69957.2us
I20260709 13:12:07.327986 32748 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:12:07.328009 32706 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:07.349797 32706 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:07.235949 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.342986 32025 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:07.349315 29008 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:07.352552 29009 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:07.361078 29156 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:07.350260 32706 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:07.362772   322 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:07.363591 32025 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:07.366147 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 163086us
I20260709 13:12:07.383548 29001 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:07.379066 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:07.376196 29708 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:07.386857 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.407475 29002 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:07.378568 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 72217.2us
I20260709 13:12:07.395395 32723 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:07.408932 32723 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.409723 32748 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:12:07.410338 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.407565 29051 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:07.411854 32748 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:07.412278 32748 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:07.400134 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.412680 32736 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:07.413244 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46254.7us
I20260709 13:12:07.409924 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.414140 29122 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.414896 32692 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:07.415128 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.415314 32692 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:07.415692 32692 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:07.416461 32574 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:07.417062 32574 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:07.418193 32574 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.415970 32735 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:07.421041 32735 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:07.421465 32735 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:07.425948   395 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:07.376449 32737 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.429886   346 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.430306 32737 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.430465   346 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.360013   320 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.376276 29010 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:12:07.433326 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 87239.4us
I20260709 13:12:07.412617 32748 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:07.439596 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 64748.9us
I20260709 13:12:07.444892 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.447817 29155 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:07.449164 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:07.469288 32596 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:12:07.470722 32596 raft_consensus.cc:3055] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:07.471141 32596 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:07.447999 28902 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:07.473666 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.474818 32696 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:07.474408 28902 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:07.475327 32696 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:07.475801 32696 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:07.477232 32737 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:07.481411 28904 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52660 (request call id 247) took 38920 ms (client timeout 15000 ms). Trace:
I20260709 13:12:07.440753 32736 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:07.395893   402 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.486846 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:07.486961 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 48984.7us
I20260709 13:12:07.484803 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.490878 29006 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:07.492012 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 286148us
I20260709 13:12:07.488842 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:07.511438 32552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.009s	sys 0.008s
I20260709 13:12:07.517251 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.483209   399 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:12:07.490602 32737 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:07.542550 32568 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:12:07.543555 28975 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.546165 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.544222 32570 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:07.547744 32570 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:07.548308 32570 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:07.545228 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.549310 32570 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.544063   358 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:12:07.551335 32736 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:07.552304 32736 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:07.552395 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 113382us
I20260709 13:12:07.552716 32736 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:07.539281 32737 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:07.553331 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.545225 32667 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.553550 32737 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:07.544955 32676 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:07.533176 32552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.009s	sys 0.009s
I20260709 13:12:07.564400 32667 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.567083 32667 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.568650 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.569820 29005 raft_consensus.cc:2410] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 9 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 13:12:07.519619 32723 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:07.583979 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 95198.8us
W20260709 13:12:07.589239 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 50818.4us
I20260709 13:12:07.587014 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:07.518354   400 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:12:07.587491 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:07.592428 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 236050us
I20260709 13:12:07.540912   346 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:07.491516 32596 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.607127 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:07.605072 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 167895us
I20260709 13:12:07.588338   320 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:07.550365 28902 catalog_manager.cc:5697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:07.619500 32705 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:07.604811 32676 raft_consensus.cc:697] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:07.620123   400 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:07.623121 32705 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.625440 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:07.612408 32723 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.628756 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.629575 29002 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:07.630816   320 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:07.633487   320 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:07.636277   320 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:07.636562 32676 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.638778 32596 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.644632 32630 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:07.645304 32630 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.645876 32630 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.645232 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:07.482012 28904 rpcz_store.cc:269] 0709 13:11:28.564086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.587376 (+ 23290us) service_pool.cc:224] Handling call
0709 13:12:07.481094 (+38893718us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:00.230394 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:00.234029 (+  3635us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:00.234081 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:12:00.234102 (+    21us) tablet.cc:662] Decoding operations
0709 13:12:00.256336 (+ 22234us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:00.256438 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:00.256466 (+    28us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:12:00.276769 (+ 20303us) tablet.cc:701] Row locks acquired
0709 13:12:00.276823 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:12:00.276993 (+   170us) write_op.cc:270] Start()
0709 13:12:00.277147 (+   154us) write_op.cc:276] Timestamp: P: 1783602720276930 usec, L: 0
0709 13:12:00.277173 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:12:00.277932 (+   759us) log.cc:844] Serialized 4873 byte log entry
0709 13:12:01.542055 (+1264123us) op_driver.cc:464] REPLICATION: finished
0709 13:12:02.242283 (+700228us) write_op.cc:301] APPLY: starting
0709 13:12:02.242427 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:02.242813 (+   386us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:02.285045 (+ 42232us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:02.744987 (+459942us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:02.745334 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:12:02.745384 (+    50us) write_op.cc:312] APPLY: finished
0709 13:12:02.745764 (+   380us) log.cc:844] Serialized 89 byte log entry
0709 13:12:02.875581 (+129817us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:02.905161 (+ 29580us) write_op.cc:454] Released schema lock
0709 13:12:02.928035 (+ 22874us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:03.036665 (+108630us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:03.039406 (+  2741us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":883655,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":676542,"apply.run_cpu_time_us":16264,"apply.run_wall_time_us":733346,"num_ops":12,"prepare.queue_time_us":149214,"prepare.run_cpu_time_us":57253,"prepare.run_wall_time_us":242985,"raft.queue_time_us":746767,"raft.run_cpu_time_us":30057,"raft.run_wall_time_us":757130,"replication_time_us":1264689,"thread_start_us":1523303,"threads_started":5,"wal-append.queue_time_us":428135,"wal-append.run_cpu_time_us":45408,"wal-append.run_wall_time_us":2859674}]]}
I20260709 13:12:07.652918 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:07.663209 32727 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:07.655652 32705 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.663926 32727 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:07.659578 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.654342 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.661077 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.626300   400 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:07.656908 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.671301 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:07.677063 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.699384 29002 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:07.701568 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:07.656183 28902 catalog_manager.cc:5697] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 5 to 7, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 7 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:07.660338   321 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:12:07.722177 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 62136.9us
I20260709 13:12:07.661717 32597 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.648036 32630 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:07.693011 32354 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:07.702008 32667 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:07.727063 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.653996 32706 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.682353 32666 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:07.700958 32723 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:07.824188 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 301137us
I20260709 13:12:07.759702   321 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:07.730834   402 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:07.831261 29710 raft_consensus.cc:2410] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 9 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 13:12:07.824504 32689 raft_consensus.cc:3055] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:07.847878 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.821022 32666 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:07.767004 32696 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.827129 32667 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:07.835928   321 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:07.867470 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 321346us
I20260709 13:12:07.878589 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.879750 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.880414 32025 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:07.881603 28975 leader_election.cc:400] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8.
I20260709 13:12:07.882416 28975 leader_election.cc:403] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:12:07.883664 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.850260 32689 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:07.888671 29151 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:07.842592 32727 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:07.855556 32666 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:07.870332 32568 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:07.904893 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:07.846906 32597 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:07.952508 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:07.952639 32689 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.953917 32689 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:07.957050 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 403815us
W20260709 13:12:07.958323 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52386.1us
W20260709 13:12:07.961016 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 85137.9us
I20260709 13:12:07.961876 28975 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:07.967736 29714 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:07.968900 32671 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.969486 32671 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:07.969990 32727 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:07.971520 32671 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:07.973861 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2347) took 2324 ms. Trace:
W20260709 13:12:07.974090   418 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:12:07.974259 29152 rpcz_store.cc:276] 0709 13:12:05.649375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.347485 (+698110us) service_pool.cc:224] Handling call
0709 13:12:06.377302 (+ 29817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:06.384781 (+  7479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.384827 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:07.220488 (+835661us) spinlock_profiling.cc:243] Waited 40.9 ms on lock 0x7b18002ef7e0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a
0709 13:12:07.222119 (+  1631us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b2c0012c0f8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:12:07.258842 (+ 36723us) log.cc:844] Serialized 26 byte log entry
0709 13:12:07.590649 (+331807us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:07.594820 (+  4171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.594903 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.973519 (+378616us) raft_consensus.cc:1685] finished
0709 13:12:07.973549 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.973670 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10590,"raft.queue_time_us":9712,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":27107,"spinlock_wait_cycles":100418304,"thread_start_us":331404,"threads_started":1,"wal-append.queue_time_us":608686}
I20260709 13:12:07.976878 29708 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:07.980854 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.891240 32630 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:07.990586 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 153389us
I20260709 13:12:07.999468 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:07.954897 32568 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:08.029011 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:08.030164 32571 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:08.030601 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:08.030982 32676 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:08.031960   358 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:08.006340 32757 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.718s	user 0.008s	sys 0.004s
I20260709 13:12:08.042992 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:08.005357 32574 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:08.050582 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 215786us
W20260709 13:12:08.033304   420 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.258s	user 0.002s	sys 0.000s
I20260709 13:12:08.010511   395 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:08.034031 32603 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:08.057901 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 58506.2us
W20260709 13:12:08.055418 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.875s (SENT) [suppressed 25 similar messages]
I20260709 13:12:07.883700   400 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.030614 32571 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:08.080358 32696 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:08.080811 32571 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.081991 32571 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:08.083477 32748 consensus_meta.cc:287] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.619s	user 0.005s	sys 0.004s
I20260709 13:12:08.083889 32748 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:08.086103 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.086597   395 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.064831 32667 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:08.089329 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica befda4a6c8344c26b0f139e4e1686b3b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:08.068846 32603 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.033175   424 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8.
I20260709 13:12:08.067369 29683 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:08.075959 32706 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:08.076156   448 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:08.054427 28902 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 5 to 6, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 6 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.077450   400 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:08.092374 32603 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:08.085918 32748 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:08.017109   320 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.057791   402 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.122133 29005 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:08.122021 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 110985us
W20260709 13:12:08.124676 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:08.107933 32735 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.686s	user 0.011s	sys 0.001s
W20260709 13:12:08.098403   419 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:12:08.076014   444 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:08.097440   441 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:08.151376 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:08.146572 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:08.158249 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:12:08.168943 29048 leader_election.cc:336] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.617s (SENT)
I20260709 13:12:08.169211 32597 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:08.169508   320 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:08.169698   341 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:08.170117   341 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.170499   341 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:08.172286 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:08.172916 29050 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:08.142287 32596 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:08.170425 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 119122us
I20260709 13:12:08.169474 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:08.191058 29003 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:08.153144   448 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:08.193424 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:08.143836 32735 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:08.274578 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.145291   402 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.190053 32671 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.208s
I20260709 13:12:08.189035 32568 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.444941 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:08.510253 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.521260 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:08.505141 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.483115 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:08.281840 29710 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:12:08.450906 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 295496us
I20260709 13:12:08.506402 29079 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:08.454556 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 625) took 1281 ms. Trace:
W20260709 13:12:08.598990 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 154128us
I20260709 13:12:08.601603 29005 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:08.417718 29716 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:12:08.611106 29716 pending_rounds.cc:85] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Aborting all ops after (but not including) 7
I20260709 13:12:08.611429 29716 pending_rounds.cc:107] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:12:08.611742 29716 raft_consensus.cc:2889] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:08.194932 32723 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:08.504616 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:08.506898 29084 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:12:08.527835 32737 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.955s	user 0.009s	sys 0.001s
I20260709 13:12:08.507541 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.605254 29083 raft_consensus.cc:2393] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:08.425662 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 223915us
W20260709 13:12:08.495267   440 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:12:08.508177 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:08.546406 32692 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.130s	user 0.009s	sys 0.002s
W20260709 13:12:08.535995 32666 consensus_meta.cc:287] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.000s
W20260709 13:12:08.514016   321 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.623s	user 0.008s	sys 0.000s
W20260709 13:12:08.453353 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 284385us
I20260709 13:12:08.631296 29082 raft_consensus.cc:2393] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:08.272128 32597 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:08.501085   452 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:12:08.298784 32696 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:08.513849 32735 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:08.426539 32748 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:08.478643   462 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.608520 29078 rpcz_store.cc:276] 0709 13:12:07.172909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.176169 (+  3260us) service_pool.cc:224] Handling call
0709 13:12:07.176571 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:07.177046 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.177074 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:07.177691 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 13:12:07.498755 (+321064us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:07.521069 (+ 22314us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.774557 (+253488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.775058 (+   501us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:08.454298 (+679240us) raft_consensus.cc:1685] finished
0709 13:12:08.454326 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:08.454487 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34336,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":30742,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":2732,"thread_start_us":573641,"threads_started":2,"wal-append.queue_time_us":719641}
I20260709 13:12:08.192437 32596 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:08.444016 32354 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:08.613575 32736 consensus_meta.cc:287] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.060s	user 0.006s	sys 0.007s
I20260709 13:12:08.419624 32564 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.627911 29085 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:08.629766 29080 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:08.395912 32667 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:08.627964 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:08.631836 29077 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:08.636199 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:08.638906 29081 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:08.628424 32737 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.662249 32737 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:08.632984 32666 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.304946   441 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:08.640172   465 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:08.667925   465 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:08.668875   441 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:08.669447   453 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:08.669564 32596 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:08.670033   465 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:08.670220   455 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:08.670822   455 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:08.671515 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.672154 29080 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:08.673565 32564 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:08.656371 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.541311 28902 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.657809 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.632081 32692 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.660254 29079 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:08.303771   448 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:08.676781 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2090) took 1114 ms. Trace:
I20260709 13:12:08.680680 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.677505 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:08.689777 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.673179 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.633837   321 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:08.691666 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.639490 32597 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.695086 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.696116 29077 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:08.688019 29001 rpcz_store.cc:276] 0709 13:12:07.562246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.570794 (+  8548us) service_pool.cc:224] Handling call
0709 13:12:07.571235 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:07.571654 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.571702 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:07.572334 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 13:12:07.831314 (+258980us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:07.831352 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.831427 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:08.676583 (+845156us) raft_consensus.cc:1685] finished
0709 13:12:08.676609 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:08.676725 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1504,"raft.queue_time_us":138708,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":17687,"thread_start_us":258517,"threads_started":1,"wal-append.queue_time_us":578949}
W20260709 13:12:08.696631 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:08.697371 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63390.9us
I20260709 13:12:08.697609 29080 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:08.676023 32564 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:08.691807 32571 consensus_meta.cc:287] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.000s
I20260709 13:12:08.703456 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:08.705354 29083 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:08.656657 32736 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.708683 29708 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:08.660033 32696 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.706465 29078 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:08.701642 29082 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:08.711978 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:08.716408 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:08.716902 29083 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:08.688024 32354 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.698486 32667 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:08.714051 32574 consensus_meta.cc:287] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.656s	user 0.001s	sys 0.010s
I20260709 13:12:08.721681 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:08.701908   441 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.676122   462 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:08.723259 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.712482 32666 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:08.715121 32696 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:08.721810   457 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.281s	user 0.000s	sys 0.002s
I20260709 13:12:08.730161 29080 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:08.730016 29077 raft_consensus.cc:2410] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:12:08.726274 32689 consensus_meta.cc:287] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.740s	user 0.003s	sys 0.008s
I20260709 13:12:08.753334 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:08.754159 29082 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:08.755905 29123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.031s	sys 0.052s
W20260709 13:12:08.756259 29123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.032s	sys 0.052s
W20260709 13:12:08.759676 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 82254.3us
I20260709 13:12:08.765712   455 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:08.728240 32574 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:08.724440 32737 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:12:08.801445   473 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.758988 32723 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:08.727845   460 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:12:08.813967 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:08.816238 29077 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
W20260709 13:12:08.803990   437 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.747s	user 0.019s	sys 0.014s
I20260709 13:12:08.818837 29080 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:12:08.821175 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56461us
I20260709 13:12:08.823091 29125 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:08.731248   441 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:08.793978   402 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:08.839432 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46625.2us
W20260709 13:12:08.846204 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 94482.8us
I20260709 13:12:08.849987 32574 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:08.850453 32574 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.850798 32574 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:08.855950 29082 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
W20260709 13:12:08.857446   471 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:12:09.005340 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 157879us
W20260709 13:12:08.874570   466 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:12:09.042296 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.043968 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.050442 29003 raft_consensus.cc:2410] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:12:09.052361 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 206191us
I20260709 13:12:09.053336 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:08.876245   402 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:09.058514 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:08.873884 32736 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:08.857448   441 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:08.813005   473 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.073329 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:09.076321 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:09.073992 32603 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.926s	user 0.013s	sys 0.006s
I20260709 13:12:09.081427   468 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.081584 32603 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.082149   468 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.084249   467 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.084968   467 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.085376 32603 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:09.088266   467 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.089954   450 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.090739   450 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.093657 32666 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:09.110487   473 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:09.118847 29125 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:09.053898   321 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:09.053225 32597 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.123461 29005 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
W20260709 13:12:09.124567 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.131824 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:09.117359   460 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:09.123472   466 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:12:09.125674   341 consensus_meta.cc:287] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.954s	user 0.000s	sys 0.020s
I20260709 13:12:09.140172 29003 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:09.122656 32727 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:09.142108 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:09.142670 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:09.148615 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 142415us
I20260709 13:12:09.126325 32354 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.126828 32736 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:09.080907   402 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.155030 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:09.137815 32597 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:09.133885   460 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.144685   462 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:09.160516 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:09.162914 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:09.035277 28902 catalog_manager.cc:5697] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:09.170367 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:09.171990 29152 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:09.160516 32570 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:09.161468   460 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:09.161620 32354 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:09.149391   341 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.187237 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:09.187750 29083 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:09.187721 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.188879 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.180194 32568 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:09.190825 29714 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:09.191072 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.194943 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.195539 29083 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:09.190909 32568 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.198544 32568 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:09.198972 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.199342 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:09.199605 29083 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:09.184106 32736 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:12:09.201205   468 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:09.201936 32603 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:09.202725 32603 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:09.204227 32667 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:09.190166   455 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.206297   455 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.206638 29083 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:09.208717   455 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.212778 32568 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.204027 32603 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:12:09.215472 32603 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:09.215732 32568 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.213505   341 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.169428 32727 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:12:09.217201 28902 catalog_manager.cc:5697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:09.218839 28902 catalog_manager.cc:5697] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 7, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 7 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:12:09.216298 29048 leader_election.cc:336] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.670s (SENT)
I20260709 13:12:09.215869 32603 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:09.220810 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:09.222019 32603 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.223438   441 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:09.184708 32354 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.224294 32354 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:09.178085 32570 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.228296 32570 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:09.230508 29708 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.002s
I20260709 13:12:09.230856 29708 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:09.232803   395 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.233203   402 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:09.234176   400 consensus_meta.cc:287] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.106s	user 0.010s	sys 0.001s
I20260709 13:12:09.235258 29085 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:12:09.235636 29085 pending_rounds.cc:85] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 7
I20260709 13:12:09.235888 29085 pending_rounds.cc:107] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:12:09.236207 29085 raft_consensus.cc:2889] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:09.186587 32666 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:09.242027 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:09.244167 29156 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:09.247246 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:09.267792 29079 consensus_meta.cc:287] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.581s	user 0.014s	sys 0.002s
I20260709 13:12:09.268957 29079 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:09.262858 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.271237 29157 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:09.274578 29077 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:09.275491 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.276214 29156 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:09.273326 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 88445us
I20260709 13:12:09.240410 32574 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.282961 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.283730 29159 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:09.278363 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 76941.8us
I20260709 13:12:09.283731 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.286682 29151 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:12:09.249090 32696 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.294458 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:09.295147 29158 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:09.285075 32574 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.310837 29157 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:09.308506 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:09.312707   402 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:09.313088   402 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.313423   402 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:09.318526 32570 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.319617 29152 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:09.329221 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:09.331362 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:09.333470 32570 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:09.338912 32723 consensus_meta.cc:287] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.000s
I20260709 13:12:09.345264 32723 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:09.343931 29680 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:09.347043   467 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:09.347569   467 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:09.348834   322 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:09.349603 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.350162 29081 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:09.350976   467 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.10, Last appended by leader: 10, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.353875   450 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.335630   460 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.355648   448 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.358088 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.359162   460 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.362635   402 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.370781   402 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:09.329738   432 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.203s	user 0.008s	sys 0.002s
W20260709 13:12:09.377938 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.359228 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.392867 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:09.391876 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.395026 29716 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:09.322769 32354 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.389175 32705 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.328814 28902 catalog_manager.cc:5697] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 7, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:09.409346 29153 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:09.399681 32571 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.426379 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:09.422307 32571 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.429294 32354 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.404099 32705 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.438112 32705 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:09.438476 32603 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:09.440448 29048 leader_election.cc:336] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.972s (SENT)
I20260709 13:12:09.442405 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:09.453080 29081 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:09.454813 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.455538 29081 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:09.458212   423 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.356s	user 0.002s	sys 0.011s
I20260709 13:12:09.458531   453 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:09.458997   453 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.459360   453 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:09.459373   429 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 1.362s	user 0.006s	sys 0.013s
I20260709 13:12:09.461534 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 628) took 2231 ms. Trace:
I20260709 13:12:09.461872 29076 rpcz_store.cc:276] 0709 13:12:07.230104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.230345 (+   241us) service_pool.cc:224] Handling call
0709 13:12:07.230743 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:07.258288 (+ 27545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.258329 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:07.304718 (+ 46389us) log.cc:844] Serialized 26 byte log entry
0709 13:12:07.581322 (+276604us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:07.581386 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.581483 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.458841 (+1877358us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:09.461142 (+  2301us) raft_consensus.cc:1685] finished
0709 13:12:09.461163 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.461361 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1240,"raft.queue_time_us":22490,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":153,"spinlock_wait_cycles":16797568,"thread_start_us":276085,"threads_started":1,"wal-append.queue_time_us":737464}
I20260709 13:12:09.463637 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.464334 29076 raft_consensus.cc:2368] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10.
I20260709 13:12:09.465088 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.465557 29076 raft_consensus.cc:2376] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10. Re-sending same reply.
I20260709 13:12:09.466286 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.466756 29076 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:09.471447   486 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.472115   486 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.476082 29076 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:09.477029 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.469345   402 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.477834   402 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.478363 28902 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 7, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 7 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:09.479866   402 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:09.480739   447 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 1.382s	user 0.000s	sys 0.009s
I20260709 13:12:09.477819 29076 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:09.480417 28902 catalog_manager.cc:5697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.482807 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.483407 29084 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:09.469671 32597 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:09.487974 29076 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:09.498394 29076 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:09.502352 29155 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:09.503016 29076 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:09.508785 29076 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:09.508896 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:09.509829 29005 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:09.510977 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.510107 29076 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.504793   486 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.505568 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:09.507381 29001 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:09.517622 29007 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:09.508344 29003 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:09.511701 29004 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
W20260709 13:12:09.508121 29048 leader_election.cc:336] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.015s (SENT)
I20260709 13:12:09.483160 28902 catalog_manager.cc:5697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.519742 29048 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:09.522055 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.525733   462 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:09.527613   462 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:12:09.529134 29048 leader_election.cc:336] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.049s (SENT)
I20260709 13:12:09.529734 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:09.530619   448 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:09.545619 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.546826 29005 raft_consensus.cc:2410] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:12:09.548681 32025 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.551256 29076 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:09.552788 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:09.554503 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:09.555287 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.556983 29084 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:09.555765 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:09.558833 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.562940 29006 raft_consensus.cc:2393] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:09.564109 29007 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:09.565732 29004 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:09.566197 29008 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:09.564457 32025 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.569172 32564 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:09.569960 32564 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.572477 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:09.572839 32597 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.573453 32597 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.574024 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:09.576932 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.578377 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.578464 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:09.579008 29083 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:09.579118 29007 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:09.583945 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:09.585793 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.585875 29081 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:09.587507 29076 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:09.588246   464 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:09.588815   464 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.590917   464 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.593400 32597 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.595003 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.597260 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.595508 32564 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.553016 32354 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.600389 32354 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:09.600966 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.602322 32354 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.603379 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.610340 29710 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
I20260709 13:12:09.604902 29007 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
W20260709 13:12:09.608840 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 43609.2us
I20260709 13:12:09.611398 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:09.620355 29122 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:09.623314   451 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.525s	user 0.005s	sys 0.005s
I20260709 13:12:09.625941 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:09.621685 32354 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:09.628105 32025 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.628281 32354 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.599772   453 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.628430 32630 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:09.628790 32354 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:09.623345   421 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.523s	user 0.018s	sys 0.007s
I20260709 13:12:09.623420   493 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.633736   478 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:09.636799   478 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:09.637193   478 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:09.635192 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:09.604679 28902 catalog_manager.cc:5697] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 10, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:09.644026 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 114914us
I20260709 13:12:09.648799 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:09.641144 32025 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:12:09.649369 29048 leader_election.cc:336] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.051s (SENT)
I20260709 13:12:09.649731 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:09.660583 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.657575 32025 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:12:09.666380 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.667444 29076 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:09.670380 32025 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:09.670898 32025 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:09.669297 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.662645   486 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.673727   486 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.680766 29001 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:09.682957 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.684253 29003 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:12:09.683187 32025 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:09.666705 28976 connection.cc:256] RPC call timeout handler was delayed by -1.7367228609999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 13:12:09.684952 28976 connection.cc:256] RPC call timeout handler was delayed by -1.4793228610000142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 13:12:09.685325 28976 connection.cc:256] RPC call timeout handler was delayed by -1.1697228609999684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260709 13:12:09.685600 28976 connection.cc:256] RPC call timeout handler was delayed by -1.1445228610000413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
I20260709 13:12:09.665011 32727 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:09.683241   486 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.664716   395 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:09.648022   460 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:12:09.668991   427 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.658293 32723 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:12:09.698283 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1891) took 1071 ms. Trace:
I20260709 13:12:09.704803 32025 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:09.695925   399 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:09.670782   448 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:09.705168 29085 rpcz_store.cc:276] 0709 13:12:08.627363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.699648 (+ 72285us) service_pool.cc:224] Handling call
0709 13:12:08.714696 (+ 15048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.245012 (+530316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.698001 (+452989us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b540034e754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:09.698210 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195991424}
I20260709 13:12:09.664113   493 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.699106   395 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:09.707300 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 57960.8us
W20260709 13:12:09.715701 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.683601 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.693542 28902 catalog_manager.cc:5697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 8. New cstate: current_term: 8 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.717451   395 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:09.716403 29048 leader_election.cc:336] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.112s (SENT)
I20260709 13:12:09.721562 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:09.704110 32727 raft_consensus.cc:697] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:09.727623 29010 raft_consensus.cc:2410] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:09.707826   399 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:09.738732 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:09.708436   427 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.732275   448 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:09.743856 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:09.744974 29710 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:09.745980 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:09.744427   453 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:09.774120 29708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47384 (request call id 773) took 1630 ms (client timeout 1587 ms). Trace:
W20260709 13:12:09.774565 29708 rpcz_store.cc:269] 0709 13:12:08.143207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.143653 (+   446us) service_pool.cc:224] Handling call
0709 13:12:09.773925 (+1630272us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b5400502154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:09.774018 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264040704}
I20260709 13:12:09.757942 32564 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.777215 32564 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.779081 32571 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:09.779258 32564 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.781255 32727 consensus_queue.cc:237] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.782493 32723 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:09.783033 32723 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.784376 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.784961 32723 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:09.787312 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.788194 29158 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:09.788558 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:09.789258 29150 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:09.785004 29156 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:09.789992   427 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:09.790551 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.791260   493 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:09.793362   448 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.793879   448 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:09.795743   448 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:09.796217   500 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.797073 29156 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:09.796476 28902 catalog_manager.cc:5697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.797731 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.797659 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.820538 29716 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:09.799216 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.799404 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.821617 29080 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 4.
I20260709 13:12:09.800165 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:09.800418 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.802819   462 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.805513 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1898) took 1142 ms. Trace:
I20260709 13:12:09.807411 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:09.811277 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:09.812445 29122 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:09.818150 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 72236us
I20260709 13:12:09.818817 29710 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:09.758527 32603 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:09.827085 29085 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:09.803927   395 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:09.827831 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 65880us
I20260709 13:12:09.831579 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.829344 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:09.825623 29008 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:09.824194 29077 rpcz_store.cc:276] 0709 13:12:08.662805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.831858 (+169053us) service_pool.cc:224] Handling call
0709 13:12:09.269618 (+437760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.274957 (+  5339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.805292 (+530335us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54001adbd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:09.805456 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949921664}
I20260709 13:12:09.828786 29152 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:09.761291   428 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.660s	user 0.002s	sys 0.011s
I20260709 13:12:09.751832   399 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.833878 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:09.838472 29157 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:12:09.831491   494 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.006s
I20260709 13:12:09.844195 32025 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:09.859460 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:09.864524 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:09.905648   485 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:12:09.914353 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 49936.1us
I20260709 13:12:09.912614 29150 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:09.865962   346 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:09.837595   500 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:09.985863 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.830890   462 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:09.837877 32597 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:09.900583   450 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:09.989784 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.016381 32025 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:10.084829 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.079447 29152 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:10.083663 29048 leader_election.cc:336] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.516s (SENT)
W20260709 13:12:10.022516 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 199146us
I20260709 13:12:10.077891 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1631) took 2962 ms. Trace:
I20260709 13:12:10.044648 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:10.000415   346 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:10.096966 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.097666 29155 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:10.095970 29154 rpcz_store.cc:276] 0709 13:12:07.080772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.099746 (+ 18974us) service_pool.cc:224] Handling call
0709 13:12:07.189820 (+ 90074us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:07.303000 (+113180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.303036 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:07.303586 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:12:07.595938 (+292352us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:07.595982 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.596070 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.705819 (+2109749us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:10.019649 (+313830us) raft_consensus.cc:1685] finished
0709 13:12:10.019669 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.042878 (+ 23209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3796,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":15639808,"thread_start_us":279365,"threads_started":1,"wal-append.queue_time_us":796734}
I20260709 13:12:10.000274 32676 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:10.136158 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:10.053298   503 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.161s	user 0.004s	sys 0.000s
W20260709 13:12:10.076318   501 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:12:10.136287 32025 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.052655 32570 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:10.132848 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.141816 29001 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:10.001039 32723 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:10.076646 32597 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:10.099932 29158 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:10.148263 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 64630.5us
I20260709 13:12:10.094521   469 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.099113   346 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:10.154881   507 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:10.156312   507 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:12:10.160195 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.028442 32630 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:10.161530 32630 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:10.164229 32354 consensus_meta.cc:287] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.003s
I20260709 13:12:10.165002 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.164824 32354 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.165779 29077 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:10.167047 32354 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:10.168308 28902 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 9, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.170078 28902 catalog_manager.cc:5697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 7, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 7 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.171267 29001 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:10.172305 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.173028 29077 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:10.174422   441 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.174535 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.174952   441 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:10.177731   500 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:10.178547 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 243002us
I20260709 13:12:10.179527   399 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:10.179682 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:10.180727   427 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:10.181187 32571 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:10.181495   493 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:10.181926 32571 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:10.182330 32571 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:10.077498   478 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:10.186367 29048 leader_election.cc:336] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.521s (SENT)
I20260709 13:12:10.181264 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:10.043054   448 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:12:10.190804 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.191383 29005 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.188283 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.193432 32571 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.194736 32571 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:10.194917 32696 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:10.195658   504 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.098s	user 0.000s	sys 0.004s
W20260709 13:12:10.193116 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 93640.2us
I20260709 13:12:10.197870 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1910) took 1021 ms. Trace:
I20260709 13:12:10.199908 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:10.199571 29051 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:10.149488 32692 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:10.202819 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:10.206537 29158 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:10.199205 29082 rpcz_store.cc:276] 0709 13:12:09.176465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.260121 (+ 83656us) service_pool.cc:224] Handling call
0709 13:12:09.284382 (+ 24261us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:09.291956 (+  7574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.292001 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:09.292612 (+   611us) log.cc:844] Serialized 52 byte log entry
0709 13:12:09.616464 (+323852us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:09.630114 (+ 13650us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.632019 (+  1905us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.634616 (+  2597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.634731 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.777563 (+142832us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b18000f9700. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:12:09.900326 (+122763us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18000f9700. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112754522 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:12:10.172727 (+272401us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18000f9700. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411a76aaa1 00007f411a761b37 00007f411a7619ca 00007f411a85926f
0709 13:12:10.174194 (+  1467us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b480015c258. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411a76aaa1 00007f411a761b37 00007f411a7619ca 00007f411a85926f 00007f411a858f37 00007f411a86d55c 00007f411a86d508 00007f411a86d488 00007f411a86d443 00007f411a86c6ec 00007f4119796a96
0709 13:12:10.197654 (+ 23460us) raft_consensus.cc:1685] finished
0709 13:12:10.197685 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.197812 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29320,"prepare.queue_time_us":44679,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":1063,"raft.queue_time_us":22432,"raft.run_cpu_time_us":9256,"raft.run_wall_time_us":548801,"spinlock_wait_cycles":64035968,"thread_start_us":238938,"threads_started":1,"wal-append.queue_time_us":348156}
I20260709 13:12:10.207971 29157 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:10.209254 29156 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:10.210795 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.211393 29085 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:10.212616   448 consensus_queue.cc:1243] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:10.213228   508 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:10.213609   508 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:10.214380   508 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:10.215528   508 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:10.219524 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:10.222601 29085 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:10.104749   450 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.225943 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:10.226485 29715 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.227041   500 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:10.229673 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:10.231467   448 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:10.231870   448 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:10.232244   448 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:10.203821   395 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:10.219600 32603 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:10.236439 32603 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:10.236862 32603 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:10.206593 32354 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:10.243233 32354 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:10.243625 32354 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:10.249082 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:10.254195 29005 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:12:10.256534 29008 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 10. Preceding OpId from leader: term: 8 index: 11. (index mismatch)
I20260709 13:12:10.257800   500 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:10.258756   500 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.263089 32630 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:10.264576 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 83404.4us
I20260709 13:12:10.265803 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:10.264456 29150 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:10.266485   493 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:10.267273 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.269289 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.269462 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.272818   448 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.273746 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.280566 29715 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:12:10.142875 32676 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.189625   469 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.287286 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.288246 29150 raft_consensus.cc:2410] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:12:10.144232 32570 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.291261 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.296877 29715 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:12:10.284878 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:10.297676 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.300184 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:10.303478 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 117134us
W20260709 13:12:10.308310 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.308652 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:10.309234 29715 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:10.310761 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:10.311628 29082 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:10.313421 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.314215 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.315245 29006 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:10.324098 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.326931 29715 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:10.336424 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.337241 29005 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:10.297394   507 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:10.331390 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:10.297487 32692 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:10.171725 28902 catalog_manager.cc:5697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 7, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 7 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:10.332921 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:10.349602 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 123228us
I20260709 13:12:10.343519   453 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:10.344990   441 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:10.340034 32597 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:10.346220   462 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:10.346843 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.342830 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.303215   450 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:10.325856 32564 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:10.350591 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:10.355077 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:10.356514 29048 leader_election.cc:336] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.700s (SENT)
I20260709 13:12:10.357303 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.323482 32689 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:12:10.384752 29150 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:10.389048 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.390197 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.390497 29003 raft_consensus.cc:1217] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Deduplicated request from leader. Original: 9.7->[10.8-10.8]   Dedup: 10.8->[]
I20260709 13:12:10.391213 29003 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:12:10.391104 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.391871 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.351930   507 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:10.393853 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.329732 32696 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:12:10.397632 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.397151 29010 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:12:10.402275 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.403772 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:10.402634 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52123.7us
I20260709 13:12:10.406137 29008 raft_consensus.cc:2368] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:12:10.406718 29010 pending_rounds.cc:85] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 8
I20260709 13:12:10.406975 29010 pending_rounds.cc:107] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:12:10.407172 29010 raft_consensus.cc:2889] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:10.398307 29005 raft_consensus.cc:2368] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
I20260709 13:12:10.401932 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.408772 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.409158 29007 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:10.412503 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:10.411135 29002 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 13:12:10.411399 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.414264 32568 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:10.414642 29150 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:10.418984 29007 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:10.383345   453 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:10.425721 29156 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 11. (index mismatch)
I20260709 13:12:10.428243 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:10.376330 32692 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.432026 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.423061 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:10.433204 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 76715.1us
I20260709 13:12:10.434957   346 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:10.437410 29002 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:10.446650 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:10.447642 29156 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:10.415566   507 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:10.445574 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:10.419807 32564 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:10.456341 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:10.454401 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.461735 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.456946 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:10.421588 32568 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:10.441731 32630 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:10.463989   441 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:12:10.463554   512 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:10.461329 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.462033 29082 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:10.465855 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:10.441434 32574 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:10.468638 32705 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:10.468765 32630 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:10.473153 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:10.514725 29048 leader_election.cc:336] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.875s (SENT)
I20260709 13:12:10.491008 29077 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:10.506421 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.470604 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:10.537623 29002 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:10.551224 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 80682.3us
I20260709 13:12:10.552438 29079 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:10.557152 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.559712 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.553915 32628 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:10.561575 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.563076 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:10.565757 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51082.4us
W20260709 13:12:10.569514 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.571339 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.576432 32596 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:10.561322 32628 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:10.578320 32628 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:10.579087 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 592144us
I20260709 13:12:10.589377 32570 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:10.589941   450 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:10.590093   469 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:10.590380   450 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:10.592605 32564 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:10.468979   462 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:12:10.473629   474 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:12:10.547132 32630 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:10.603762 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.604055 32025 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:10.604557 29008 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:10.604836 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 139037us
I20260709 13:12:10.591362   450 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.572976   358 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:10.474140   400 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:12:10.602766 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:10.551280   448 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:10.629617 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.630219 29076 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:12:10.632244 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.982s (SENT)
I20260709 13:12:10.632663 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:10.469301   470 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.136s
I20260709 13:12:10.611721 29152 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:10.615164 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.612427 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.619200 32025 raft_consensus.cc:697] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:10.623108 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:10.613230 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.611362 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.642516 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:10.643220 29158 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:10.609912 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:10.609632 32706 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:10.620523   358 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:10.618999 29125 leader_election.cc:400] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10.
I20260709 13:12:10.655437 29125 leader_election.cc:403] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:12:10.627579 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.573277   478 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:10.639104 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.650789 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:10.651261 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 54183.1us
I20260709 13:12:10.658010 28975 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:10.651727 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.653566 32025 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.654318 29005 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:10.660008 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.650316 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:10.667416 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.668255 29076 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:10.668094 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.669351 29151 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 7: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:10.671108 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.671386 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.673921 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.675786 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.677098 29005 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:10.683241 29079 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:12:10.683245 29002 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:10.686363 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 69964.2us
I20260709 13:12:10.687541 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:10.686373 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.667016 29159 raft_consensus.cc:2376] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10. Re-sending same reply.
I20260709 13:12:10.690693 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.668916 29008 raft_consensus.cc:2376] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7. Re-sending same reply.
I20260709 13:12:10.692384 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.699029 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:10.699519 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:10.702185   341 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:10.702711   341 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.707424   341 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:10.707563 29005 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:10.608675 32628 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.713369   462 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:10.715029 32628 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:10.715251 32692 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:10.470676 28902 catalog_manager.cc:5697] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 8, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:10.573415   453 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.657646   486 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:10.718658   486 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:10.719007   486 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:10.721130 29081 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:10.670177   358 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:10.721900 32354 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:10.722259 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.607384   507 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.669606 32706 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.730127 32706 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:10.696965   474 raft_consensus.cc:3055] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:10.734550 29002 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:10.735777 29157 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:10.735920 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.739919 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:10.740257 28892 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 06ae4197fd784525b695a9b16c8094ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:10.741961 29001 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 12. Ops: []
I20260709 13:12:10.742202 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.745994 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.746693 29002 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:10.750460 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.748528 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.730907   474 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:10.775712 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.776973 29009 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:10.778015 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.778591 29005 raft_consensus.cc:2376] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7. Re-sending same reply.
I20260709 13:12:10.748415 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.768997 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:10.785864 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.786669 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.787389 29005 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:10.772995 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.777061 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:10.692245 32571 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:10.775213 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:10.787364 29083 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:10.766541 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 134323us
I20260709 13:12:10.789512 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:10.790066 29048 leader_election.cc:336] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.984s (SENT)
I20260709 13:12:10.698174 32737 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10.
W20260709 13:12:10.770044 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71123us
I20260709 13:12:10.786090 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.798255 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.743078   395 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:10.801260 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:10.766876 32727 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:10.802546 29158 raft_consensus.cc:2376] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11. Re-sending same reply.
W20260709 13:12:10.800519 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 221905us
I20260709 13:12:10.642414 32568 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:10.776603   504 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:12:10.791190   515 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
W20260709 13:12:10.806345 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 121149us
I20260709 13:12:10.798188   346 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:10.808435 29008 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:12:10.768210 32354 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:10.805169 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:10.859436 29048 leader_election.cc:336] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.060s (SENT)
I20260709 13:12:10.859849 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:10.860072 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:10.860620 29005 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:10.787766 32696 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:10.860764 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.849614 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:10.733502   427 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
W20260709 13:12:10.851570 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45685.2us
W20260709 13:12:10.850248   518 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:12:10.798355 32597 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:10.862974 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:10.878280 29003 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:12:10.911972 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 52575.2us
W20260709 13:12:10.895164 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.897253 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:10.781817   493 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:12:10.785565   464 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:10.907826 29005 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:10.922740 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:10.910156 29083 raft_consensus.cc:2468] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:10.925463 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:10.924489 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:10.903920 32696 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.924953   358 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:10.933923 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:10.934335 29715 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:10.871619   500 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:10.935669 29084 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:10.938961 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:10.810804 32737 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:12:10.898505   395 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:10.951813 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 143742us
W20260709 13:12:10.927932   504 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:12:10.953346 32354 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:10.953949 29125 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:10.955116 32737 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:10.859005 32570 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:12:10.930294 32597 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:11.028273 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:11.038141 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:10.923198   528 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:11.031056 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 92203.3us
I20260709 13:12:10.950928 32692 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:11.028182 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:11.035451 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 81558us
I20260709 13:12:10.939774 32727 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:10.952750   464 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:11.059720 29714 raft_consensus.cc:2368] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:12:11.074867 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:11.028293   399 consensus_meta.cc:287] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.848s	user 0.009s	sys 0.002s
I20260709 13:12:11.063973 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.085462 29077 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:11.064172   358 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:11.068738 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.072098 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.072742   395 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:11.045006 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 191666us
I20260709 13:12:11.050817 32571 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:11.074150 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:11.074834 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.075230 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:11.075922 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:11.076481 32676 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:11.076751   527 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:12:11.076923   534 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.077922 32564 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:11.113966 29085 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:11.078383 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.078876 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.081635 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:11.082571 32597 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:11.083709   525 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.417s	user 0.001s	sys 0.000s
I20260709 13:12:10.934536 28902 catalog_manager.cc:5697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 5 to 7, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 7 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:10.911425   474 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.059818 32723 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:11.096273 29048 leader_election.cc:400] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
I20260709 13:12:11.125309 29048 leader_election.cc:403] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:12:11.032215   462 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:11.153991 32696 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:11.154752 32689 consensus_queue.cc:1175] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "befda4a6c8344c26b0f139e4e1686b3b" responder_term: 8 status { last_received { term: 5 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:11.126804   534 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.157943   535 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:12:11.158939 29157 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:11.159175   535 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:11.109225 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.159658   535 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:11.160238 29156 raft_consensus.cc:2410] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:12:11.160526   535 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.161109 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.161767   535 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:11.161811 29682 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:11.162174 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:11.122411 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.085547 29708 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:12:11.125023 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.087930 29683 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.163461 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:11.126538 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44984.8us
I20260709 13:12:11.164155 29152 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:11.039628   512 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:11.129032 28902 catalog_manager.cc:5697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 4, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 4 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:11.030694 32737 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:11.113024   441 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:12:11.134517   460 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:11.134534   537 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:11.136431 32727 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:11.139065 32603 consensus_meta.cc:287] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.902s	user 0.010s	sys 0.000s
I20260709 13:12:11.108289 32571 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.141251   531 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:12:11.143157 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.148303 29680 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:11.148572 32723 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:11.200451 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 52215us
I20260709 13:12:11.148347   474 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:11.155390 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.158478   534 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:11.127480 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 49157.5us
I20260709 13:12:11.124969 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:11.215982 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:11.128767 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:11.168614 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 80770.9us
I20260709 13:12:11.132797   464 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:11.217227 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.217629 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:11.218364 29154 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:11.169432 28975 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:11.125857   529 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:12:11.169721 32689 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:11.184324 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:11.217227   523 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:11.220466 29156 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:11.168481   527 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:12:11.095032   528 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.196640 29004 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:11.226133 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.196753 29001 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:12:11.218746 29151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
W20260709 13:12:11.229048 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.177591 32705 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:11.198029 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 128822us
I20260709 13:12:11.231979 29154 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:11.190754   500 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9.
W20260709 13:12:11.227617 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 59466.8us
W20260709 13:12:11.196137   400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.004s	sys 0.021s
I20260709 13:12:11.186506 29085 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:11.197558   537 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:11.269034 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:11.199355   531 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:11.282297 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56232.9us
I20260709 13:12:11.282049 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.227337 32689 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:11.282749 29708 raft_consensus.cc:2368] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
W20260709 13:12:11.235327   400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.005s	sys 0.022s
W20260709 13:12:11.285037 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51930.7us
I20260709 13:12:11.285629 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.201170 32603 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.198868   493 consensus_queue.cc:1243] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:11.232786   523 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:11.298880 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:11.294626   507 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.298617   399 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.304358 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:11.308333 32630 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.665s	user 0.007s	sys 0.003s
I20260709 13:12:11.308745 32630 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.292752 28902 catalog_manager.cc:5697] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:11.309743 29158 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:11.234189   453 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:12:11.310045 28902 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 6, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 6 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:11.312845 32630 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:11.312709   523 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.233637   536 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:11.341297   536 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:11.341671   537 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.342254   536 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.345479   450 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:11.347301 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:11.347929 29715 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:11.348489 29717 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:11.349736 29005 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:11.351442 29152 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:11.315214 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.304373   399 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.293303   346 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
W20260709 13:12:11.318266   517 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.846s	user 0.005s	sys 0.008s
I20260709 13:12:11.319406   533 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:11.282542   542 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:12:11.357405   533 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:11.325291   528 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:11.330691   508 consensus_meta.cc:287] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.115s	user 0.011s	sys 0.000s
I20260709 13:12:11.331215 32723 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.358502   533 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.360535 29080 raft_consensus.cc:2410] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:11.332211   512 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.332481   531 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.283895 32736 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:11.311667 28902 catalog_manager.cc:5697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 6, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 6 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:11.283732 32737 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:11.338032 32574 consensus_meta.cc:287] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.801s	user 0.009s	sys 0.013s
W20260709 13:12:11.339463   486 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.000s
I20260709 13:12:11.339905 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1936) took 1049 ms. Trace:
W20260709 13:12:11.340752   543 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:12:11.355198 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 73229.3us
I20260709 13:12:11.364156   545 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:11.372972   508 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:11.367564 32737 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:11.354326   493 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:11.371650 32630 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.373063   531 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:11.366817   486 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.369309 28902 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 8, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 8 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:11.364877 32736 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:11.380784 32736 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:11.367851 32574 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:11.394258 29006 raft_consensus.cc:2368] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
W20260709 13:12:11.361805 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46635.2us
I20260709 13:12:11.366964 29085 rpcz_store.cc:276] 0709 13:12:10.290867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.291060 (+   193us) service_pool.cc:224] Handling call
0709 13:12:11.339697 (+1048637us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54000456d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:11.339821 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874663808}
I20260709 13:12:11.393141 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.395160 28976 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.396346 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:11.398511 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 180689us
I20260709 13:12:11.399446 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:11.404356 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.423405 29051 leader_election.cc:400] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
I20260709 13:12:11.423801 29051 leader_election.cc:403] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:11.399962 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.385692   508 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:11.433033 29158 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:12:11.436515 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 50644.6us
I20260709 13:12:11.442601 29155 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:12:11.446700 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:11.394695   460 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:11.421329 32564 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:11.420595   545 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:11.424069   478 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.485961 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:11.486922 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 86759.1us
I20260709 13:12:11.487937 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.430603   528 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:11.467845 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:11.468792 29077 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:11.469357 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:11.477280 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 136767us
I20260709 13:12:11.479862 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:11.430456   346 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:11.425722   465 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:11.394941 32630 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.508796 29085 raft_consensus.cc:2410] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:12:11.519125 29150 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:11.432233   542 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:12:11.520013 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.462103 29157 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:11.432163   508 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:11.508836 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.523387 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.525964 29009 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:12:11.525856 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 126488us
I20260709 13:12:11.530201 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.489818 32628 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:11.537036 29079 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:11.505692   493 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.539391 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.537611 29151 raft_consensus.cc:606] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.510985   346 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.544348 29156 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:11.519690 32571 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:11.549026 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.550293 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:11.551220 29717 raft_consensus.cc:2368] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13.
W20260709 13:12:11.552030 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 114690us
I20260709 13:12:11.552847 29048 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.553865 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:11.554593 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 132144us
I20260709 13:12:11.540450   545 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:11.537453   536 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:11.510351   460 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.548065   346 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:11.580112 29682 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:11.547853 32628 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:11.619817 29683 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:11.544548   530 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:11.621927 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 123888us
I20260709 13:12:11.556376   486 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:11.577353   528 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.551499   544 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:11.634462   544 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:12:11.635241   548 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:12:11.637430 32596 raft_consensus.cc:2804] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:11.638057 32596 raft_consensus.cc:697] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:11.579638   399 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:11.619817   515 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:12:11.645437   539 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.646050   460 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:11.648473   561 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:11.648942   561 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.649295   561 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:11.546516   493 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.651862 29157 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:12:11.654964 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 105998us
I20260709 13:12:11.655766 28976 leader_election.cc:400] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:12:11.556710 32597 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
W20260709 13:12:11.659425 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 105629us
W20260709 13:12:11.661312 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 75159us
I20260709 13:12:11.656059 28976 leader_election.cc:403] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:11.695045 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.698253 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:11.617079 32696 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:11.696012 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:11.698812 29076 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:11.630328   536 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:11.705700 29079 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:11.720728 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.707159   540 consensus_queue.cc:1059] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.149s
I20260709 13:12:11.708074   460 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:11.708561 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1486) took 1352 ms. Trace:
I20260709 13:12:11.739861 29080 rpcz_store.cc:276] 0709 13:12:10.355846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.355961 (+   115us) service_pool.cc:224] Handling call
0709 13:12:11.708370 (+1352409us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b5400146c54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:11.708507 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2229353344}
I20260709 13:12:11.740912 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:11.741207 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:11.741492 29080 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:11.742250 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:11.712677   553 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:12:11.716686   399 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.717285 32692 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:11.718590   526 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.686s	user 0.004s	sys 0.000s
I20260709 13:12:11.718200 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.719180   453 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:11.722441 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:11.753811 29083 raft_consensus.cc:2376] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11. Re-sending same reply.
I20260709 13:12:11.754894 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:11.755178 29079 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:11.727468 29076 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:11.730026 29050 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.758244 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.579886   508 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.734596 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:11.742851 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 162850us
I20260709 13:12:11.748814 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.760353 29050 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.764493 29076 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:12:11.766307 29153 raft_consensus.cc:2368] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
I20260709 13:12:11.770606 29081 raft_consensus.cc:2410] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:12:11.771296 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:11.771723 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.776094 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:11.775033 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:11.675436 32723 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:12:11.777060 29009 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:11.734113   441 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:11.777441 29082 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:11.778306 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.779047   399 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.779325 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:11.779935 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.784665 29080 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:11.785578 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:11.703444   530 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 5. Current term is 6.
I20260709 13:12:11.753708   507 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:11.814293 28892 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 06ae4197fd784525b695a9b16c8094ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:11.811620 32736 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.812510   546 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:11.816479 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:11.646047   539 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:11.775204   400 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.754282 32692 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:11.834477 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 56219.9us
W20260709 13:12:11.815790   557 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:12:11.836861 29153 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:11.835889 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 216166us
I20260709 13:12:11.789203   500 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:11.790855   450 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:11.765949   427 consensus_queue.cc:1059] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:12:11.735980 32596 consensus_queue.cc:237] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.814867   508 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:11.841869 29154 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:11.788235 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:11.858271 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:11.860881 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 72715.8us
I20260709 13:12:11.862506 29005 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:11.861775 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:11.864439 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 74684.4us
I20260709 13:12:11.869881 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:11.768281 32628 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.871789 29050 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:11.790711   441 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:11.846061   558 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:12:11.872750 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.841037 32630 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:11.842356   523 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:11.879738 29050 leader_election.cc:400] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
I20260709 13:12:11.895682 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:11.863978   400 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.909411 29159 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:12:11.905524 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1651) took 1296 ms. Trace:
I20260709 13:12:11.911789 29157 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:11.869083   448 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:11.909864 29150 rpcz_store.cc:276] 0709 13:12:10.609517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.609639 (+   122us) service_pool.cc:224] Handling call
0709 13:12:11.905357 (+1295718us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54002b57d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:11.905470 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849285504}
I20260709 13:12:11.914139 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:11.876451   500 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:11.918442 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 53167.3us
W20260709 13:12:11.941272   570 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:12:11.940833 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:11.942843 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 70161.4us
I20260709 13:12:11.849900   546 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:12:11.944204 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:11.942571 29079 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:11.909281   528 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:11.907330   399 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.925531   512 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:12:11.989447 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 73845.2us
I20260709 13:12:11.915292   448 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:11.894127 32696 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
W20260709 13:12:11.980626 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 107235us
I20260709 13:12:11.922603   450 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:12:11.986984 29152 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.001s	sys 0.010s
W20260709 13:12:11.995513 29152 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.002s	sys 0.010s
I20260709 13:12:11.946139   493 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:11.908993   523 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:11.953243   341 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:12.005611 32676 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:11.953416   400 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:12:11.989183   507 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:12.062572 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 132457us
I20260709 13:12:11.979525   528 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:12.065613 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 121475us
W20260709 13:12:12.041793   568 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
W20260709 13:12:12.055775 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.991041   399 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.071396 29079 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:12.072225 29153 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:12:12.077944 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 163858us
I20260709 13:12:12.083024 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 530) took 1003 ms. Trace:
W20260709 13:12:12.065908   531 consensus_meta.cc:287] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.014s
W20260709 13:12:12.084767 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 137495us
I20260709 13:12:12.083439 29155 rpcz_store.cc:276] 0709 13:12:11.079951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.443342 (+363391us) service_pool.cc:224] Handling call
0709 13:12:11.443564 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.489687 (+ 46123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:11.489730 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:11.489789 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:11.489839 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.082816 (+592977us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b54003ed5d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:12.082865 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:12.082974 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105482112}
I20260709 13:12:12.108222 29005 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:12.062356   530 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:12.110723   582 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:12:12.117362   579 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:12:12.112185   581 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:12:12.116367 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:12.118764 29079 raft_consensus.cc:2410] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:12.117609   582 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:12.121377   523 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.121842   582 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.088328   400 consensus_queue.cc:1243] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:12.123121   582 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:12.124339   448 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.125592   448 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:12.128340 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2384) took 1383 ms. Trace:
I20260709 13:12:12.128750 29159 rpcz_store.cc:276] 0709 13:12:10.744655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.354902 (+610247us) service_pool.cc:224] Handling call
0709 13:12:11.905940 (+551038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:11.914676 (+  8736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.128139 (+213463us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54002b57d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:12.128279 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1196227328}
I20260709 13:12:12.129050   530 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.131428   530 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.134486 32603 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:12.080971   528 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:12.142797 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2374) took 1527 ms. Trace:
I20260709 13:12:12.143267 29156 rpcz_store.cc:276] 0709 13:12:10.615003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.221010 (+606007us) service_pool.cc:224] Handling call
0709 13:12:11.221266 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.547659 (+326393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.142559 (+594900us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b5400054e54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:12.142734 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742599424}
W20260709 13:12:12.145701   584 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:12:12.150631   341 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:12.160269   441 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.157500   528 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.168255 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:12.176736 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:12.199393   528 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.204659 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:12.082271   500 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:12.208737 29085 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:12.212116 29077 raft_consensus.cc:1240] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 7. Current term is 8. Ops: [6.7-7.8]
I20260709 13:12:12.091671 32736 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:12.231407 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:12.127295   561 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:12.131968   346 consensus_meta.cc:287] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.002s
I20260709 13:12:12.239514 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:12.219807   590 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:12.249500 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:12.250036 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:12.252375 29077 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:12.253690 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:12.219049   539 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:12.256913 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.241264 32696 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:12.264602 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.249732   582 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:12.265268 29010 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:12.220482   500 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.259546   441 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:12.271922 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.272727 29010 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:12.244374   346 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:12.280424 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.255971   590 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.283601   533 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:12.284124   590 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.289996 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:12.290895 29008 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:12.291648   542 consensus_meta.cc:287] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.763s	user 0.000s	sys 0.010s
I20260709 13:12:12.274497 32696 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.305478 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:12.311707 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.298940   542 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
I20260709 13:12:12.301111   594 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:12.320581   594 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.322645   594 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.322801 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.323144   594 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.323835 29004 raft_consensus.cc:2410] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:12:12.307056 32737 consensus_meta.cc:287] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.933s	user 0.010s	sys 0.003s
I20260709 13:12:12.326079 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:12.329327   448 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:12.329800   448 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:12.330204   448 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:12.314883 32696 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.303683   582 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.334348 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.323592   594 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:12.335927 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.338544   500 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.332618   399 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.340783   582 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.342767   582 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.345280   561 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.346024 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:12.347036 29082 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:12.333530   528 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:12.349956   346 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:12.353478   594 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:12.356352 32696 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:12.359145 29081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 637) took 1640 ms (client timeout 1892 ms). Trace:
I20260709 13:12:12.360441 29083 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:12:12.360814 29083 pending_rounds.cc:85] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 7
I20260709 13:12:12.361016 29083 pending_rounds.cc:107] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:12:12.361222 29083 raft_consensus.cc:2889] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:12.363464 29082 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:12.366418 29008 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
W20260709 13:12:12.359542 29081 rpcz_store.cc:269] 0709 13:12:10.718860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.722028 (+  3168us) service_pool.cc:224] Handling call
0709 13:12:12.358950 (+1636922us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540034f154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:12.359090 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407171840}
W20260709 13:12:12.377456 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:12.381083 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:12.376621 29683 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:12.388873 29010 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:12:12.385433   508 consensus_meta.cc:287] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.001s
I20260709 13:12:12.393172   341 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:12.395779 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:12.397789 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:12.399708 29683 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:12.420562 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:12.435029 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:12.413666   531 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:12.438201   572 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:12.438822   572 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:12.439709   572 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.414093   523 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:12.413836   508 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:12.412302   533 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:12.460582   523 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:12.423169 32603 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:12:12.460628   508 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:12.461628   523 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:12.463006   508 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.464654 29004 raft_consensus.cc:3055] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:12.465031 29004 raft_consensus.cc:740] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:12.472801 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.473071 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.473407 29008 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:12.471077 32603 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:12:12.474993 32603 consensus_queue.cc:1243] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:12.465904 29004 consensus_queue.cc:260] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:12.481464 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 523) took 1765 ms. Trace:
I20260709 13:12:12.481905 29154 rpcz_store.cc:276] 0709 13:12:10.716243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.234713 (+518470us) service_pool.cc:224] Handling call
0709 13:12:11.281859 (+ 47146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.843390 (+561531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.481220 (+637830us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54003edad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:12.481402 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287442816}
I20260709 13:12:12.483465 32692 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.498226   493 consensus_queue.cc:1059] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:12.502323 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 526) took 1777 ms. Trace:
I20260709 13:12:12.504163   538 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:12.504788   538 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:12.510197 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 213479us
I20260709 13:12:12.512923 29005 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:12.510102   450 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:12.515254 32025 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:12.516263 32564 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:12.516629 32603 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.516700 32564 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:12.517267 32603 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.517776 32564 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:12.519138 32564 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:12.519364 32603 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.471928   533 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:12:12.520808 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:12.521202 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52580 (request call id 20) took 1442 ms. Trace:
I20260709 13:12:12.521517 32737 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:12.521975 32737 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:12.522459 32737 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:12.469830   531 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:12.523419   531 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:12.523808   531 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:12.521869 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.525612 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:12.525902 32025 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:12.526788 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.527377 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.529695 29008 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:12.530715 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.533430 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.535760 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.537307 29083 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:12.537649 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.521631 29151 rpcz_store.cc:276] 0709 13:12:11.079091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.218602 (+139511us) service_pool.cc:224] Handling call
0709 13:12:12.521044 (+1302442us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400054e54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583b853 00007f411a80960a 00007f4113d8f7d4 00007f4113d8f762 00007f4113d8f691 00007f4113d8f60c 00007f4113d8e8c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:12.521133 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":708760448}
I20260709 13:12:12.539700 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:12.541535 29076 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:12.543078 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:12.543582 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:12.544171 29083 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:12.545779 29717 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:12:12.546149 29717 pending_rounds.cc:85] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Aborting all ops after (but not including) 6
I20260709 13:12:12.546371 29717 pending_rounds.cc:107] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:12:12.546595 29717 raft_consensus.cc:2889] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:12.502823 29152 rpcz_store.cc:276] 0709 13:12:10.724793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.352095 (+627302us) service_pool.cc:224] Handling call
0709 13:12:11.352410 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:11.363398 (+ 10988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:11.363428 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:11.390971 (+ 27543us) log.cc:844] Serialized 52 byte log entry
0709 13:12:12.011238 (+620267us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:12.062740 (+ 51502us) log.cc:844] Serialized 14 byte log entry
0709 13:12:12.063127 (+   387us) log.cc:844] Serialized 14 byte log entry
0709 13:12:12.501636 (+438509us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b180048cfa0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:12:12.501828 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.501924 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:12.502058 (+   134us) raft_consensus.cc:1685] finished
0709 13:12:12.502078 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:12.502253 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19176,"prepare.queue_time_us":79779,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":2019,"raft.queue_time_us":455949,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":159,"spinlock_wait_cycles":23240704,"thread_start_us":553340,"threads_started":1,"wal-append.queue_time_us":691882}
I20260709 13:12:12.407027   448 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.534055 29157 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:12.549567 29004 raft_consensus.cc:3060] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:12.550546 29076 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:12.551710 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.552079 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.557322 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:12.558143 29076 raft_consensus.cc:2393] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 10: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:12.559554 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.560703 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.561213 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:12.561851 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:12.562741 29085 raft_consensus.cc:2410] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:12.563310 29001 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:12.564940 29150 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:12.572264 29084 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:12.572928 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:12.573561 29152 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:12.579106 29006 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:12:12.579557 29006 pending_rounds.cc:85] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 4
I20260709 13:12:12.579798 29006 pending_rounds.cc:107] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:12:12.580076 29006 raft_consensus.cc:2889] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:12.580755 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:12.581367 29153 raft_consensus.cc:2376] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9. Re-sending same reply.
I20260709 13:12:12.581532 29714 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:12:12.582576 29153 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:12.584892 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:12.589390 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:12.590545 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:12.594252 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:12.597544 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.598250 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:12.600350 29077 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:12.603516 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:12.603806 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.561110   448 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:12.614662 29078 pending_rounds.cc:85] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 4
I20260709 13:12:12.615135 29078 pending_rounds.cc:107] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:12:12.615456 29078 raft_consensus.cc:2889] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:12.591187   441 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.631151 29152 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:12.606019   594 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:12.605698   399 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.591365 32705 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:12:12.724993 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.725672 29152 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:12.601140 32628 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:12:12.601796 32696 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 13:12:12.701189 29083 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 5.
W20260709 13:12:12.710784 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:12.698990 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 66037.7us
I20260709 13:12:12.699824   594 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:12.719563   341 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.820696 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:12.636874   534 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 13:12:12.828452 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:12.662554   441 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.699067 32568 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.113s
W20260709 13:12:12.840018   554 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:12:12.863490 32568 consensus_queue.cc:1243] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:12.873751   561 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.881513   561 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:12.884406   561 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:12.885085   615 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.181s	user 0.000s	sys 0.009s
W20260709 13:12:12.891175 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 62812.2us
I20260709 13:12:12.887724 29004 raft_consensus.cc:1275] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:12:12.892127 29004 pending_rounds.cc:85] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 7
I20260709 13:12:12.892464 29004 pending_rounds.cc:107] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:12:12.892777 29004 raft_consensus.cc:2889] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:12.825789   594 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:12.895409 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:12.729038   399 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.901870 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:12.903007 29003 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:12.904361 29157 raft_consensus.cc:2393] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 10: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:12.912395 29157 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:12:12.904012   540 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:12:12.848323   341 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:12.897994 32354 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:12.934221 29083 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:12.954505 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:12.949757   572 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.926098 32568 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:12:12.963934   572 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:12.967656 29151 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:12.864960   560 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:12:13.032747   572 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.070456   460 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:13.078709   560 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:13.078644   460 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.079288   560 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:13.080363   560 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.089586 29002 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:12:13.089942 29156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.037s	sys 0.081s
W20260709 13:12:13.091473 29156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.037s	sys 0.082s
I20260709 13:12:13.154510 29682 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:13.160835 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 206158us
I20260709 13:12:13.162798 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:13.168802 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 259795us
I20260709 13:12:13.155696 28975 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:13.186375 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:13.190148 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:13.191505 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:13.195444 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:13.070747 32354 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:13.192857 32568 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:12:13.192589   508 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:13.217562 32737 consensus_meta.cc:287] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.695s	user 0.009s	sys 0.001s
I20260709 13:12:13.218286   460 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:13.222152 32603 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:13.222739 32727 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:12:13.223546   605 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.506s	user 0.000s	sys 0.001s
I20260709 13:12:13.229831 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:13.197026   539 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:12:13.250619   470 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:13.231134 32737 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.197458   530 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:12:13.232585 32354 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.236927   582 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:13.245059   399 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.193342   572 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:13.184564   561 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:13.248790   478 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:13.208125   528 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:13.253774 32603 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:13.262926 29154 raft_consensus.cc:2368] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
I20260709 13:12:13.258852   508 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:13.264065   531 consensus_meta.cc:287] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.740s	user 0.011s	sys 0.000s
I20260709 13:12:13.272522 32603 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.273206   531 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.274827   594 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.262266   561 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:13.277652   561 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:13.279554   531 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.288868 32706 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:13.289410 32706 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.285429   594 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.291472 32706 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.263850   341 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:13.336437 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 113169us
W20260709 13:12:13.339010 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 109254us
I20260709 13:12:13.343986 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:13.344816 29157 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:12:13.345971 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 159659us
I20260709 13:12:13.347692 29050 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:13.349071 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:13.319934 32737 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:13.321378   538 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:13.353746 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:13.287474   561 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.349845   508 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:13.360880 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:13.364843 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.383402 29157 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:13.361488 32354 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:13.385984 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:13.406690 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51521.8us
I20260709 13:12:13.387995 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:13.357082 32564 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:12:13.352488   531 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:12:13.362458   528 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:13.405740 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:13.417486 32025 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:13.419118 29159 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.423543 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:13.424768 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:13.425742 29153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:13.426230 29153 raft_consensus.cc:606] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:13.421844   341 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:12:13.431025 29153 raft_consensus.cc:614] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:13.434477   469 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:13.412983   500 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:13.414352 32723 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:13.437824   561 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.448757 29008 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:13.450646 29010 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
W20260709 13:12:13.459107 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 99333us
I20260709 13:12:13.465209 29010 pending_rounds.cc:85] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 6
I20260709 13:12:13.465631 29010 pending_rounds.cc:107] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:12:13.465926 29010 raft_consensus.cc:2889] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:13.450718   531 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:13.444381   500 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.473738   530 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:13.497117 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 89433.4us
I20260709 13:12:13.498329 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:13.499351 29125 leader_election.cc:400] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13.
I20260709 13:12:13.499737 29125 leader_election.cc:403] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:12:13.502211 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:13.509092 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:13.509783 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:13.517552   500 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.518299 32354 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:13.486032   531 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.488876   530 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.504765 32564 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13.
I20260709 13:12:13.504887 32692 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.504664   534 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:13.558072 29005 pending_rounds.cc:85] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:12:13.558490 29005 pending_rounds.cc:107] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:12:13.558779 29005 raft_consensus.cc:2889] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:13.547387   631 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:12:13.557547   628 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:12:13.567209 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:13.583798 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:13.584395 29010 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:13.560451   534 raft_consensus.cc:697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:13.584407 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:13.559968 32692 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.622784 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:13.626951   519 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.627732   519 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.629951 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1663) took 1091 ms. Trace:
I20260709 13:12:13.630483 29151 rpcz_store.cc:276] 0709 13:12:12.538976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.551928 (+ 12952us) service_pool.cc:224] Handling call
0709 13:12:13.629723 (+1077795us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54000942d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:13.629869 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756353792}
I20260709 13:12:13.632973 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:13.634478   633 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:13.635151   633 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:13.636922   519 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:13.631516 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:13.639848   633 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.643307   508 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:13.643847   508 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.645907   508 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:13.646384   508 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.646812   508 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.649813   508 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.652505   540 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:13.653210   540 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.657565   540 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:13.659255 29082 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:13.662128 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:13.662726 29001 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:13.632028 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:13.664207   627 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:13.665331 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2392) took 1155 ms. Trace:
I20260709 13:12:13.666193 29085 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:13.666479 32689 raft_consensus.cc:3055] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:13.666975 32689 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:13.668272 29082 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:13.668120 32689 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.669253 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:13.669734 32689 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.670497 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:13.691166   560 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:13.694505   441 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.665849 29154 rpcz_store.cc:276] 0709 13:12:12.509584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.521567 (+ 11983us) service_pool.cc:224] Handling call
0709 13:12:13.665099 (+1143532us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54003ed0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:13.665260 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1698059648}
I20260709 13:12:13.674538 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:13.675531 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:13.676213 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:13.676296 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:13.702574   500 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:13.703068   500 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.678983   632 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.703495   500 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:13.703805   632 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.705436   448 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.706121   632 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.705928   448 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:13.679332 28892 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 06ae4197fd784525b695a9b16c8094ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:13.679399   624 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:13.682482   634 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:13.684938   523 consensus_queue.cc:1048] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:13.699627 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:13.769236 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:13.593776   629 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:12:13.792479 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:13.769878 29154 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:13.772755 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:13.798122 28975 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:13.794185 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:13.798283 29151 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:13.661374   540 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:13.803201 29682 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:13.812764 29050 leader_election.cc:400] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
I20260709 13:12:13.813205 29050 leader_election.cc:403] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:13.598273 32737 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:13.821336 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:13.805974   627 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:13.817520   528 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:13.798938   634 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.805606   441 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:12:13.799602   539 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:12:13.793393   508 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:13.707979   448 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.707818   624 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.837026 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:13.793299   632 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:13.883627 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 46749.5us
W20260709 13:12:13.873064 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.806607   588 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:13.865976   460 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.005s	sys 0.018s
I20260709 13:12:13.895151 29158 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:13.830104   572 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:13.832484   636 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:13.819607   465 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:13.854497 32603 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:13.878248   531 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.828557   540 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:13.829598 29010 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:13.891518   448 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:13.894207   493 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:13.902387   460 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.005s	sys 0.019s
I20260709 13:12:13.898569   632 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.883033   508 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.869361   534 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.878576   637 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:12:13.898934   588 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.923743 32574 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:12:13.938848   465 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:13.948030   531 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:13.938571 32603 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:13.990567   530 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:13.960770   493 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:14.046591   470 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:14.047418   470 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:14.051326   470 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:14.061518   441 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:14.075781   441 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:14.079253   441 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:14.025646   530 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:14.094739   644 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:12:14.008752   531 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:14.034801   658 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:12:14.096971   567 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:14.097391   567 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:14.097492   530 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:14.098250   567 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:14.099152   618 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.881s	user 0.003s	sys 0.001s
I20260709 13:12:14.099443   567 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:14.097508 32692 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:14.115567 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2393) took 1592 ms. Trace:
I20260709 13:12:14.116076 29156 rpcz_store.cc:276] 0709 13:12:12.522712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.523063 (+   351us) service_pool.cc:224] Handling call
0709 13:12:12.523433 (+   370us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:12.523847 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:12.523879 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:12.524498 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 13:12:13.154278 (+629780us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:13.612768 (+458490us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.612949 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.613021 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:14.098635 (+485614us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:14.103194 (+  4559us) raft_consensus.cc:1685] finished
0709 13:12:14.115285 (+ 12091us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:14.115497 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":458117,"prepare.queue_time_us":463827,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":9934,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":190,"spinlock_wait_cycles":24538368,"thread_start_us":567285,"threads_started":1,"wal-append.queue_time_us":693356}
I20260709 13:12:14.063133   448 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:14.135551 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:14.131907 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.150305 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.151944 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.105722 32692 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:14.106566   469 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:14.152701 29153 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:14.097393   658 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:14.132061   646 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:12:14.103334   441 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:14.172523 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.182358 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.181324 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.153898   633 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:14.191879 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.138748   448 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:14.119951   465 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:14.258819 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:14.180349 32727 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:12:14.205214   624 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:14.180022   658 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:14.169157 32692 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:14.187765 32630 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:12:14.171726   469 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:14.239235   649 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:12:14.300410 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.182080   441 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:14.290277 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 133162us
W20260709 13:12:14.290009 28976 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.66:35939 timed out after 1.507s (SENT) [suppressed 29 similar messages]
I20260709 13:12:14.276822 29006 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:12:14.227401   633 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:14.318228 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 73612.4us
W20260709 13:12:14.550340   652 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 13:12:14.551823   668 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:14.552104   572 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:14.552712   572 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:14.553500 29714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 578) took 1032 ms. Trace:
I20260709 13:12:14.553881 29714 rpcz_store.cc:276] 0709 13:12:13.521222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.521363 (+   141us) service_pool.cc:224] Handling call
0709 13:12:13.521630 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.521963 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.521982 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:13.522284 (+   302us) log.cc:844] Serialized 26 byte log entry
0709 13:12:13.893648 (+371364us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b1800087a00. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:12:14.179876 (+286228us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:14.185191 (+  5315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:14.185260 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:14.553226 (+367966us) raft_consensus.cc:1685] finished
0709 13:12:14.553250 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:14.553440 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14696,"spinlock_wait_cycles":5485952,"thread_start_us":285977,"threads_started":1,"wal-append.queue_time_us":1027882}
I20260709 13:12:14.552389   668 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:14.475226 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:14.455734 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.557751 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.557942 29079 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:14.245668   674 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.260672   508 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:14.316838   661 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
W20260709 13:12:14.546797   651 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.454s	user 0.001s	sys 0.000s
W20260709 13:12:14.417116   646 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 13:12:14.536899   653 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.463s	user 0.001s	sys 0.000s
I20260709 13:12:14.325700   624 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:14.568071 32603 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:14.604979 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:14.605805 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:14.231460   540 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:14.325426   588 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:14.416837   671 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.548266   670 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:14.490792   493 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:14.418263   537 consensus_queue.cc:1059] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.260s
W20260709 13:12:14.634660 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.978929 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2413) took 1303 ms. Trace:
W20260709 13:12:14.636682 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 384660us
I20260709 13:12:14.755623 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:14.742197 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.195454 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:14.791884 28975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.011s	sys 0.018s
W20260709 13:12:15.199014 28975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.016s	sys 0.052s
I20260709 13:12:14.601532   682 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.722079 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2194) took 1164 ms. Trace:
W20260709 13:12:14.929593 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.801434 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 544) took 1415 ms. Trace:
I20260709 13:12:15.219799 29152 rpcz_store.cc:276] 0709 13:12:13.385932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.386046 (+   114us) service_pool.cc:224] Handling call
0709 13:12:13.386306 (+   260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.498883 (+112577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.498910 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:13.505935 (+  7025us) log.cc:844] Serialized 26 byte log entry
0709 13:12:13.581314 (+ 75379us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:13.620536 (+ 39222us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.620800 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.620898 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:14.780702 (+1159804us) raft_consensus.cc:1685] finished
0709 13:12:14.801138 (+ 20436us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:14.801352 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64770,"prepare.queue_time_us":42702,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":574,"raft.queue_time_us":35747,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":146,"thread_start_us":124878,"threads_started":2,"wal-append.queue_time_us":375431}
I20260709 13:12:14.538853   679 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:15.206790 29005 rpcz_store.cc:276] 0709 13:12:13.557268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.557429 (+   161us) service_pool.cc:224] Handling call
0709 13:12:13.557860 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.559369 (+  1509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.559411 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:13.559894 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:12:13.808377 (+248483us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:13.808410 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.808475 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:14.721769 (+913294us) raft_consensus.cc:1685] finished
0709 13:12:14.721800 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:14.721994 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":248018,"threads_started":1,"wal-append.queue_time_us":384445}
W20260709 13:12:14.698901   656 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.550s	user 0.001s	sys 0.000s
W20260709 13:12:15.156596   672 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:12:14.615151 32696 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:14.680243   453 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.000s	sys 0.011s
I20260709 13:12:15.194461 29009 rpcz_store.cc:276] 0709 13:12:13.675788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.688198 (+ 12410us) service_pool.cc:224] Handling call
0709 13:12:13.688585 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.709421 (+ 20836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.709459 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:13.709924 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:12:13.816481 (+106557us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:13.816533 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:14.978356 (+1161823us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b54001856d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:14.978430 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:14.978499 (+    69us) raft_consensus.cc:1685] finished
0709 13:12:14.978519 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:14.978831 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1109,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":47033216,"thread_start_us":106147,"threads_started":1,"wal-append.queue_time_us":241832}
I20260709 13:12:15.336018 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.336762 29079 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
W20260709 13:12:14.769326   666 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.296s	user 0.000s	sys 0.000s
W20260709 13:12:15.325150 29154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.339s	user 0.015s	sys 0.027s
I20260709 13:12:14.626600 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:15.339581 29154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.370s	user 0.015s	sys 0.027s
I20260709 13:12:15.185635   341 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:15.101301   636 consensus_meta.cc:287] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.170s	user 0.005s	sys 0.007s
W20260709 13:12:15.102277   560 consensus_meta.cc:287] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.410s	user 0.012s	sys 0.003s
W20260709 13:12:14.989375   653 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
W20260709 13:12:15.128482   567 consensus_meta.cc:287] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.029s	user 0.008s	sys 0.004s
W20260709 13:12:15.000025   664 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.682s	user 0.002s	sys 0.002s
I20260709 13:12:14.680961   678 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:15.366119   678 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.366932   441 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.368991   678 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:15.329609 29050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.000s	sys 0.010s
W20260709 13:12:15.192880   572 consensus_meta.cc:287] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.000s
I20260709 13:12:15.370360   572 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.371376 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:15.371971 29009 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:15.372304   572 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.372779   540 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:15.373360   540 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:15.373818 29050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.000s	sys 0.011s
I20260709 13:12:15.327816   470 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:15.374548   678 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:15.375284   540 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.375082   470 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.375620   441 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:15.376238   441 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.305576   679 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.378142   441 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:15.379505   679 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:15.378895   441 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.273484   682 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.380097   441 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.230911   624 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.382189   441 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:15.382517   624 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.383322   658 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:14.600548   674 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.384697 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:15.385497   674 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:15.386018   441 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:15.386384   441 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:15.386719   441 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:15.336916   453 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.348s	user 0.002s	sys 0.021s
I20260709 13:12:15.337332 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1526) took 1086 ms. Trace:
I20260709 13:12:14.606022 32692 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:15.149883 32689 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.390738 29079 rpcz_store.cc:276] 0709 13:12:14.250460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.335753 (+1085293us) service_pool.cc:224] Handling call
0709 13:12:15.337277 (+  1524us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:14.699731   632 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:15.102342   675 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.107523   668 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.231076   588 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.038398   539 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:15.340097 29080 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:15.403111 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1525) took 1153 ms. Trace:
I20260709 13:12:15.403491 29080 rpcz_store.cc:276] 0709 13:12:14.250168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.626295 (+376127us) service_pool.cc:224] Handling call
0709 13:12:15.403043 (+776748us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.231573   346 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.360554   341 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.231245   670 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.362167 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1514) took 2749 ms. Trace:
I20260709 13:12:15.362779   469 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.363253   636 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10.
I20260709 13:12:15.231331   671 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.390071 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:15.390736 29083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1516) took 2775 ms (client timeout 1507 ms). Trace:
I20260709 13:12:15.415555 29050 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:15.415791 29083 rpcz_store.cc:269] 0709 13:12:12.613567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.820347 (+206780us) service_pool.cc:224] Handling call
0709 13:12:15.388488 (+2568141us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540027b5d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:15.388633 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259038080}
I20260709 13:12:15.417193 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.417953 29083 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:15.418530 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 669) took 1135 ms. Trace:
I20260709 13:12:15.418923 29083 rpcz_store.cc:276] 0709 13:12:14.282958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.416870 (+1133912us) service_pool.cc:224] Handling call
0709 13:12:15.418462 (+  1592us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.420372 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.421334 29083 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:15.422019 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1522) took 1172 ms. Trace:
I20260709 13:12:15.422446 29083 rpcz_store.cc:276] 0709 13:12:14.249430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.419673 (+1170243us) service_pool.cc:224] Handling call
0709 13:12:15.421955 (+  2282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.423614 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:15.424817 29083 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:14.324887   500 consensus_meta.cc:287] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.621s	user 0.000s	sys 0.017s
W20260709 13:12:15.255208 28976 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.006s	sys 0.000s
W20260709 13:12:15.425701 28976 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.010s	sys 0.000s
W20260709 13:12:15.426175 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 1.13608e+06us
W20260709 13:12:15.427008 28976 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Timed out: RequestConsensusVote RPC to 127.27.63.66:35939 timed out after 1.634s (SENT)
I20260709 13:12:15.427574 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:15.249668 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:15.393603 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.428892 29079 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:15.429525 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1527) took 1178 ms. Trace:
I20260709 13:12:15.429919 29079 rpcz_store.cc:276] 0709 13:12:14.250723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.393315 (+1142592us) service_pool.cc:224] Handling call
0709 13:12:15.429453 (+ 36138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.374439   633 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:15.396240 32689 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.398304   675 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:15.393646   656 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:12:15.388244   470 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.389899   658 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:15.406574 29004 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.104s	user 0.028s	sys 0.058s
I20260709 13:12:15.392422   624 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:15.412467 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.407907 29078 rpcz_store.cc:276] 0709 13:12:12.613064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.613906 (+   842us) service_pool.cc:224] Handling call
0709 13:12:12.614398 (+   492us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:12.634033 (+ 19635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:12.634065 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:12.634592 (+   527us) log.cc:844] Serialized 52 byte log entry
0709 13:12:14.296392 (+1661800us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b18003e0800. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:12:14.599663 (+303271us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:14.599729 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:14.599911 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:15.361888 (+761977us) raft_consensus.cc:1685] finished
0709 13:12:15.361909 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.362107 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":431,"raft.queue_time_us":11459,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":16531,"spinlock_wait_cycles":20931328,"thread_start_us":303009,"threads_started":1,"wal-append.queue_time_us":2641324}
I20260709 13:12:15.482758 29080 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:15.483299 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1521) took 1234 ms. Trace:
I20260709 13:12:15.375303   678 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.431221 29156 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:15.486150 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.485145 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1688) took 1208 ms. Trace:
W20260709 13:12:15.406363 32727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.020s	sys 0.006s
I20260709 13:12:15.445093 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.406935   588 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.447881   633 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:15.455163 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:15.460539 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:15.494364 28973 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.010s	sys 0.001s
I20260709 13:12:15.461864 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:15.462087 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:15.462361 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:15.406662   539 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.468055   668 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:15.479427 29004 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.177s	user 0.029s	sys 0.058s
W20260709 13:12:15.480087 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 95044.9us
W20260709 13:12:15.481506 29682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.004s	sys 0.025s
I20260709 13:12:15.483690 29080 rpcz_store.cc:276] 0709 13:12:14.249189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.404203 (+1155014us) service_pool.cc:224] Handling call
0709 13:12:15.483237 (+ 79034us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.486941 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1523) took 1237 ms. Trace:
I20260709 13:12:15.487442 29158 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:15.487485 29156 rpcz_store.cc:276] 0709 13:12:14.277103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.561440 (+284337us) service_pool.cc:224] Handling call
0709 13:12:15.485029 (+923589us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.489344 29079 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:12:15.441545 29122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.013s	sys 0.001s
W20260709 13:12:15.660522 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 288268us
W20260709 13:12:15.661976 29122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.166s	user 0.013s	sys 0.001s
I20260709 13:12:15.661986 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1524) took 1412 ms. Trace:
W20260709 13:12:15.662423 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 1.18761e+06us
I20260709 13:12:15.447002 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 674) took 1142 ms. Trace:
I20260709 13:12:15.432753   469 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:15.408744   346 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.467916   692 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:15.495396 29716 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:15.442525   594 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:15.444000 32692 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:15.494781 28973 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.010s	sys 0.001s
I20260709 13:12:15.497184 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:15.488731 32727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.021s	sys 0.006s
I20260709 13:12:15.498070 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:15.667196 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2203) took 1432 ms. Trace:
I20260709 13:12:15.578548 29077 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:15.491904   624 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.631270   635 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:15.500450   697 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:15.506595 29078 rpcz_store.cc:276] 0709 13:12:14.249673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.482987 (+1233314us) service_pool.cc:224] Handling call
0709 13:12:15.486887 (+  3900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.633216   657 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.552274   453 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:15.585214 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:15.487807   658 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.482641 32564 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:15.642412 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.446389   500 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:15.544752   656 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:12:15.490170   689 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:15.554085   668 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.666632   469 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.720904   715 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:15.721483   469 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.721431   715 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.722394 32689 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.723534   715 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.724762 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1515) took 3111 ms. Trace:
I20260709 13:12:15.714210 32564 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.727062 29008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:43948
W20260709 13:12:15.506210 29682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.004s	sys 0.025s
I20260709 13:12:15.662510 29079 rpcz_store.cc:276] 0709 13:12:14.249919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.444569 (+1194650us) service_pool.cc:224] Handling call
0709 13:12:15.661930 (+217361us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.662985 29083 rpcz_store.cc:276] 0709 13:12:14.304803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.423341 (+1118538us) service_pool.cc:224] Handling call
0709 13:12:15.446931 (+ 23590us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.665954 29001 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
W20260709 13:12:15.668614 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 468591us
I20260709 13:12:15.667395 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:15.667542 29004 rpcz_store.cc:276] 0709 13:12:14.234980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.241832 (+  6852us) service_pool.cc:224] Handling call
0709 13:12:14.242190 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:14.242455 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:15.500585 (+1258130us) log.cc:844] Serialized 14 byte log entry
0709 13:12:15.666876 (+166291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.666928 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.666988 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.667045 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.667147 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1300248,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":43,"thread_start_us":1318376,"threads_started":2}
W20260709 13:12:15.699486 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 114353us
W20260709 13:12:15.701053 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 254422us
I20260709 13:12:15.667550   681 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:15.714468 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:15.715377 29680 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:15.698880   453 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:15.727770 29008 raft_consensus.cc:606] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:16.028650 29008 raft_consensus.cc:614] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:16.029367 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1675) took 2569 ms. Trace:
I20260709 13:12:16.029779 29154 rpcz_store.cc:276] 0709 13:12:13.460201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.798335 (+338134us) service_pool.cc:224] Handling call
0709 13:12:13.801752 (+  3417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:13.911832 (+110080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.911874 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:13.913562 (+  1688us) log.cc:844] Serialized 52 byte log entry
0709 13:12:15.340504 (+1426942us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:15.340555 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.340620 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:16.029178 (+688558us) raft_consensus.cc:1685] finished
0709 13:12:16.029195 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.029321 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98564,"raft.queue_time_us":126486,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":5797,"thread_start_us":1371094,"threads_started":1,"wal-append.queue_time_us":1547722}
I20260709 13:12:15.783174 29076 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:15.777283 29009 raft_consensus.cc:2410] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:12:16.033175 28892 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 06ae4197fd784525b695a9b16c8094ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:12:15.769793 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.725127 29081 rpcz_store.cc:276] 0709 13:12:12.613339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.240037 (+626698us) service_pool.cc:224] Handling call
0709 13:12:13.240395 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:13.240589 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:13.277661 (+ 37072us) log.cc:844] Serialized 14 byte log entry
0709 13:12:15.724281 (+2446620us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b18003e0b60. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287
0709 13:12:15.724377 (+    96us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.724435 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.724487 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.724565 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.724709 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":927,"prepare.queue_time_us":79549,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":41,"raft.queue_time_us":22645,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7618,"spinlock_wait_cycles":7242240,"thread_start_us":173376,"threads_started":3,"wal-append.queue_time_us":127984,"wal-append.run_cpu_time_us":1162,"wal-append.run_wall_time_us":1001206}
I20260709 13:12:15.729060 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2414) took 1590 ms. Trace:
W20260709 13:12:16.037792 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 322488us
I20260709 13:12:15.795686 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:16.057080 29080 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:16.039381 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:16.057642 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1528) took 1806 ms. Trace:
I20260709 13:12:15.701960   679 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:15.710269 32603 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.710474   632 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.713169   572 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:15.710566 32692 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.058267 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:15.668987   692 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:16.112679 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 445317us
I20260709 13:12:16.113178 29080 rpcz_store.cc:276] 0709 13:12:14.250981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.776168 (+1525187us) service_pool.cc:224] Handling call
0709 13:12:16.057589 (+281421us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.037088 29153 rpcz_store.cc:276] 0709 13:12:14.139021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.276195 (+1137174us) service_pool.cc:224] Handling call
0709 13:12:15.276650 (+   455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:15.293098 (+ 16448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.293160 (+    62us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:15.335972 (+ 42812us) log.cc:844] Serialized 52 byte log entry
0709 13:12:15.553553 (+217581us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:15.574489 (+ 20936us) log.cc:844] Serialized 14 byte log entry
0709 13:12:15.575601 (+  1112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.575679 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:15.728792 (+153113us) raft_consensus.cc:1685] finished
0709 13:12:15.728820 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.728994 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5807,"prepare.queue_time_us":20676,"prepare.run_cpu_time_us":7011,"prepare.run_wall_time_us":17684,"raft.queue_time_us":96813,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":37703,"thread_start_us":157413,"threads_started":1,"wal-append.queue_time_us":331929}
I20260709 13:12:15.716521   705 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:16.139372 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 81182.2us
I20260709 13:12:16.149865 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:16.151516 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.141781 29081 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:15.726883   508 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:15.698378   635 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.158200 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:15.893261   627 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:15.801000   708 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:12:15.800251   709 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:12:15.802119   691 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:12:15.801607   702 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:12:16.153437 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1980) took 2377 ms. Trace:
W20260709 13:12:16.148165 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 202225us
I20260709 13:12:15.797547   715 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.038714   674 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.155280 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:16.161254 32025 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.144809   588 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.148456   705 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.174629 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1972) took 2504 ms. Trace:
I20260709 13:12:16.054947   341 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:15.849490   540 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.054121   678 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.178531 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1983) took 2401 ms. Trace:
I20260709 13:12:16.342190 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1677) took 2880 ms. Trace:
I20260709 13:12:16.113049 32564 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:15.709002   657 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.702409   697 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.137387   534 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:16.187875 29081 rpcz_store.cc:276] 0709 13:12:13.776203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.058454 (+2282251us) service_pool.cc:224] Handling call
0709 13:12:16.141448 (+ 82994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.153122 (+ 11674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.153367 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.142881   699 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:16.194218 32025 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:15.805482   633 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:16.144098   675 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.061168   470 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:16.144603   670 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:16.262794 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46589.6us
I20260709 13:12:16.057816   681 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.172151   478 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.357064   703 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:16.357539   703 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.357358   478 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.357933   703 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:16.359107   531 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.359368   478 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:16.266433 29082 rpcz_store.cc:276] 0709 13:12:13.670391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.673527 (+  3136us) service_pool.cc:224] Handling call
0709 13:12:13.673793 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.674518 (+   725us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:13.677154 (+  2636us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.940600 (+263446us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.941931 (+  1331us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:13.948134 (+  6203us) log.cc:844] Serialized 26 byte log entry
0709 13:12:13.948341 (+   207us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:13.948599 (+   258us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.948835 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.948909 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:14.142734 (+193825us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b18002f43a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112754522 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:12:15.630932 (+1488198us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b18002f43a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:12:15.800253 (+169321us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b18002f43a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112754522 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:12:16.174390 (+374137us) raft_consensus.cc:1685] finished
0709 13:12:16.174417 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.174562 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1022,"prepare.queue_time_us":209357,"prepare.run_cpu_time_us":142,"prepare.run_wall_time_us":15786,"raft.queue_time_us":6621,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":18596,"spinlock_wait_cycles":101137536,"thread_start_us":251546,"threads_started":2,"wal-append.queue_time_us":407299}
I20260709 13:12:16.360132   721 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.360620   721 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.361042 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1985) took 2583 ms. Trace:
I20260709 13:12:16.361397 29082 rpcz_store.cc:276] 0709 13:12:13.777315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.360223 (+2582908us) service_pool.cc:224] Handling call
0709 13:12:16.360543 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.360715 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.360743 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.360785 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:16.360819 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.360872 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.360991 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.362599   721 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.363402 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1986) took 2571 ms. Trace:
I20260709 13:12:16.137854   632 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.185434   539 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:16.279112 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80492.1us
I20260709 13:12:16.144392 32692 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.339161   728 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.144759   572 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:15.775592   689 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:16.137960 32603 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.197399   704 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:16.146343   679 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:16.170491   508 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.201300   671 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.264127   700 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:16.342412 29080 rpcz_store.cc:276] 0709 13:12:13.776875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.147093 (+2370218us) service_pool.cc:224] Handling call
0709 13:12:16.177947 (+ 30854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.178150 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.178188 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.178238 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:16.178274 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.178351 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.178471 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.342620 29151 rpcz_store.cc:276] 0709 13:12:13.461615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.802562 (+340947us) service_pool.cc:224] Handling call
0709 13:12:13.864594 (+ 62032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.976535 (+111941us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:14.312155 (+335620us) log.cc:844] Serialized 14 byte log entry
0709 13:12:14.843202 (+531047us) log.cc:844] Serialized 14 byte log entry
0709 13:12:14.843395 (+   193us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:15.021666 (+178271us) log.cc:844] Serialized 26 byte log entry
0709 13:12:15.021856 (+   190us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:15.021906 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.022001 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:16.341995 (+1319994us) raft_consensus.cc:1685] finished
0709 13:12:16.342015 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.342134 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163075,"prepare.queue_time_us":113076,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":631963,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":244,"thread_start_us":552174,"threads_started":2,"wal-append.queue_time_us":989889}
I20260709 13:12:16.274467   705 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:16.183175   627 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:16.205394   674 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.337056   346 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.205193   715 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:16.336334   636 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.349391   493 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:16.328847   453 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.390533 29081 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:16.393734 32025 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.440478 29122 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:16.517880 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1984) took 2740 ms. Trace:
I20260709 13:12:16.501642   718 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:16.199460 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:16.399039   719 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:12:16.400394   717 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:12:16.340957   540 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:16.399041   716 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:12:16.366146   470 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:16.352597   678 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.390396   670 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.399178   624 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.379413   500 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.501881   721 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:16.518141 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 77754us
I20260709 13:12:16.363767 29082 rpcz_store.cc:276] 0709 13:12:13.792113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.362293 (+2570180us) service_pool.cc:224] Handling call
0709 13:12:16.362549 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.362704 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.362733 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.362770 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:16.363187 (+   417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.363238 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.363354 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:16.518636 29084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.013s	sys 0.013s
I20260709 13:12:16.518244 29081 rpcz_store.cc:276] 0709 13:12:13.777093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.348177 (+2571084us) service_pool.cc:224] Handling call
0709 13:12:16.383582 (+ 35405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.517174 (+133592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.517826 (+   652us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.508626   448 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:16.562767 29084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.013s	sys 0.013s
I20260709 13:12:16.381916   633 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:16.534122   441 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.146s	user 0.011s	sys 0.000s
I20260709 13:12:16.535732   715 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:16.528867 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 329451us
I20260709 13:12:16.541360   705 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.544492 32603 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.546590   679 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.519047   493 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:16.547995   727 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:16.549310   674 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.551133   689 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.552378   632 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.553472   697 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.555954   658 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.557512   572 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.523456 32723 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:16.523137   718 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:16.566391 32692 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.501338   728 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.561100   670 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.585530 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.563592   678 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.639679 29008 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:16.517779   636 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.560108   470 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:16.582190 28897 maintenance_manager.cc:419] P c44d1d5117734537b7202f74e0c1e920: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033341
I20260709 13:12:16.570487   692 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.585477   668 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.649798 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.576864 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:16.664274 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 87488.6us
I20260709 13:12:16.665432 32354 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:16.665933 32354 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:16.666769 32354 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.667634 32696 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:16.668217 32696 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:16.639009   705 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.590940   727 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.676061 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1981) took 2899 ms. Trace:
I20260709 13:12:16.576416   721 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:16.587977   441 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:16.615278 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:16.685951 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.686743 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1978) took 2911 ms. Trace:
I20260709 13:12:16.686786 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.676234   727 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:16.676651 29083 rpcz_store.cc:276] 0709 13:12:13.776431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.053630 (+2277199us) service_pool.cc:224] Handling call
0709 13:12:16.054090 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:16.184798 (+130708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.184828 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:16.185581 (+   753us) log.cc:844] Serialized 52 byte log entry
0709 13:12:16.507045 (+321464us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:16.507084 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.507149 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:16.675765 (+168616us) raft_consensus.cc:1685] finished
0709 13:12:16.675795 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.675968 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":291347,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":24234,"thread_start_us":298328,"threads_started":2,"wal-append.queue_time_us":261581}
I20260709 13:12:16.687211 29084 rpcz_store.cc:276] 0709 13:12:13.775711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.142662 (+366951us) service_pool.cc:224] Handling call
0709 13:12:14.142964 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:14.143364 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:14.143399 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:15.803890 (+1660491us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b2c0004c3c8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:12:15.806660 (+  2770us) log.cc:844] Serialized 26 byte log entry
0709 13:12:16.564346 (+757686us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:16.564427 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.564523 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:16.676919 (+112396us) raft_consensus.cc:1685] finished
0709 13:12:16.676940 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.686685 (+  9745us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6105,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4132,"spinlock_wait_cycles":33366784,"thread_start_us":757362,"threads_started":1,"wal-append.queue_time_us":777648}
I20260709 13:12:16.691561 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 665) took 2437 ms. Trace:
I20260709 13:12:16.592072 32603 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.567420   725 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.694630 29151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8340da85294490fb88f62f80393531f"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:16.679028 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.695227 29151 raft_consensus.cc:606] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:16.695289 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.691999 29083 rpcz_store.cc:276] 0709 13:12:14.254109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.690620 (+2436511us) service_pool.cc:224] Handling call
0709 13:12:16.690966 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.691168 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.691204 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.691259 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:16.691303 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.691371 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.691500 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.686105   679 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:16.696524 29082 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:16.686810 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.697244 29153 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:16.695503 29001 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:16.697211 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 667) took 2414 ms. Trace:
I20260709 13:12:16.698719 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.698531 29083 rpcz_store.cc:276] 0709 13:12:14.282473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.696476 (+2414003us) service_pool.cc:224] Handling call
0709 13:12:16.696729 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.696923 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.696946 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.696985 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:16.697024 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.697073 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.697158 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.700412 29150 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:16.700649 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.704188 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 668) took 2421 ms. Trace:
I20260709 13:12:16.700779 29081 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:16.702880 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:16.689545   681 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:16.704511 29083 rpcz_store.cc:276] 0709 13:12:14.282703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.703524 (+2420821us) service_pool.cc:224] Handling call
0709 13:12:16.703758 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.703896 (+   138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.703920 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.703957 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:16.703984 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.704031 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.704144 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.704857 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1686) took 2463 ms. Trace:
I20260709 13:12:16.697461 29008 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:16.770987 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.751048 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 581) took 1288 ms. Trace:
I20260709 13:12:16.765100 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 556) took 2583 ms. Trace:
I20260709 13:12:16.782737 29152 rpcz_store.cc:276] 0709 13:12:14.181630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.764278 (+2582648us) service_pool.cc:224] Handling call
0709 13:12:16.764580 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.764782 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.764818 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.764859 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:16.764896 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.764951 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.765046 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.783504 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 670) took 2500 ms. Trace:
I20260709 13:12:16.784188 29083 rpcz_store.cc:276] 0709 13:12:14.283186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.781760 (+2498574us) service_pool.cc:224] Handling call
0709 13:12:16.782876 (+  1116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.783100 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.783136 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.783194 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:16.783239 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.783307 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.783438 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.782248 29708 rpcz_store.cc:276] 0709 13:12:15.454379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.454498 (+   119us) service_pool.cc:224] Handling call
0709 13:12:15.454740 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.475434 (+ 20694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.475479 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.475531 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:15.475576 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.742003 (+1266427us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b54004b69d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:16.742071 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.742234 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46998912}
I20260709 13:12:16.766073 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 554) took 3024 ms. Trace:
I20260709 13:12:16.763876 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 555) took 2582 ms. Trace:
I20260709 13:12:16.787774 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.779491   464 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:16.789583   633 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.787534 29158 rpcz_store.cc:276] 0709 13:12:13.741388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.272868 (+1531480us) service_pool.cc:224] Handling call
0709 13:12:15.273294 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.663135 (+389841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.765811 (+1102676us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54000947d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:16.766011 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482282880}
I20260709 13:12:16.788007 29159 rpcz_store.cc:276] 0709 13:12:14.181343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.763035 (+2581692us) service_pool.cc:224] Handling call
0709 13:12:16.763352 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.763535 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.763584 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.763629 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:16.763662 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.763715 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.763823 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.756157 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.791018 29150 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:16.704602 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:16.761629 29077 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 1975) took 2988 ms (client timeout 1876 ms). Trace:
I20260709 13:12:16.789402   464 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.794309 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.781929 29155 rpcz_store.cc:276] 0709 13:12:14.241596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.629228 (+1387632us) service_pool.cc:224] Handling call
0709 13:12:15.629639 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:15.652339 (+ 22700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.652369 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:15.658422 (+  6053us) log.cc:844] Serialized 52 byte log entry
0709 13:12:15.866563 (+208141us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:15.899398 (+ 32835us) log.cc:844] Serialized 14 byte log entry
0709 13:12:16.062234 (+162836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.062350 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:16.704629 (+642279us) raft_consensus.cc:1685] finished
0709 13:12:16.704648 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.704797 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":191044,"prepare.queue_time_us":187098,"prepare.run_wall_time_us":46,"raft.queue_time_us":186539,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":5401,"spinlock_wait_cycles":35344000,"thread_start_us":225819,"threads_started":2,"wal-append.queue_time_us":495591}
W20260709 13:12:16.758962   737 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:12:16.699744   679 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:16.791970 29077 rpcz_store.cc:269] 0709 13:12:13.773112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.773257 (+   145us) service_pool.cc:224] Handling call
0709 13:12:16.761428 (+2988171us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400022954. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:16.761551 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3618510720}
I20260709 13:12:16.806130 29083 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:16.806684 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 671) took 2523 ms. Trace:
I20260709 13:12:16.806696 29077 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:16.807415 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 672) took 2523 ms. Trace:
I20260709 13:12:16.807825 29150 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:16.807730 29077 rpcz_store.cc:276] 0709 13:12:14.283632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.806306 (+2522674us) service_pool.cc:224] Handling call
0709 13:12:16.806555 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.807191 (+   636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.807366 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.807058 29083 rpcz_store.cc:276] 0709 13:12:14.283410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.786057 (+2502647us) service_pool.cc:224] Handling call
0709 13:12:16.786385 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.806469 (+ 20084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.806636 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.792737 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:16.758332   735 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:12:16.809598 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 675) took 2504 ms. Trace:
I20260709 13:12:16.792095 29001 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:16.711024 32723 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.810396 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 679) took 1759 ms. Trace:
I20260709 13:12:16.810108 29083 rpcz_store.cc:276] 0709 13:12:14.310644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.808857 (+2498213us) service_pool.cc:224] Handling call
0709 13:12:16.809174 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.809329 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.809350 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.809387 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:16.809414 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.809464 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.809546 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31488}
I20260709 13:12:16.810956 32723 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.812088 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1988) took 1429 ms. Trace:
I20260709 13:12:16.812501 29083 rpcz_store.cc:276] 0709 13:12:15.382931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.811262 (+1428331us) service_pool.cc:224] Handling call
0709 13:12:16.811526 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.811684 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.811709 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.811746 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:16.811776 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.811915 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.812008 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.813803 32723 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:16.700026   460 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:16.814353 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1989) took 1431 ms. Trace:
I20260709 13:12:16.756268   657 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:16.778616   627 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.741173   684 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:16.810905 29003 rpcz_store.cc:276] 0709 13:12:15.050995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.207322 (+156327us) service_pool.cc:224] Handling call
0709 13:12:15.305505 (+ 98183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.474211 (+168706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.474241 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.474280 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:15.474309 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.810191 (+1335882us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540035ded4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:16.810238 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.810342 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387487104}
I20260709 13:12:16.815379   684 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:16.816367 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.817382 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:16.817533   684 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.819168 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2202) took 3126 ms. Trace:
I20260709 13:12:16.819510 29002 rpcz_store.cc:276] 0709 13:12:13.693005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.699678 (+  6673us) service_pool.cc:224] Handling call
0709 13:12:13.699975 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:13.906020 (+206045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.906053 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:13.906108 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:13.906156 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.818953 (+2912797us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54000af0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:16.818997 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.819097 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473422208}
I20260709 13:12:16.820081 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.820750 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.814738   460 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.821347 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.821609 29002 raft_consensus.cc:2410] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:16.823037   460 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:16.802090 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.823305 29002 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:16.825141 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.825501 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.826208 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.826766 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.827245 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.827850 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.828269 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.828384 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.828574 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.829186 29714 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 5.
I20260709 13:12:16.814353 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1976) took 3041 ms. Trace:
I20260709 13:12:16.830461 29076 rpcz_store.cc:276] 0709 13:12:13.773416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.135079 (+361663us) service_pool.cc:224] Handling call
0709 13:12:15.388784 (+1253705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.032259 (+643475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.813538 (+781279us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b540027b5d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:16.814182 (+   644us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540027b5d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:16.814304 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3788728576}
I20260709 13:12:16.831390 29008 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:16.809598 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 673) took 2525 ms. Trace:
I20260709 13:12:16.832448 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.833508 29077 rpcz_store.cc:276] 0709 13:12:14.283874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.808849 (+2524975us) service_pool.cc:224] Handling call
0709 13:12:16.809174 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.809325 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.809348 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.809386 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:16.809414 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.809464 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.809546 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28160}
I20260709 13:12:16.834630 29008 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:16.833694 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:16.836661 29157 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:16.838521 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.699700   725 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.805691   670 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:16.702512 32603 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:16.844375 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:16.849154 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:16.841779 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1991) took 1455 ms. Trace:
I20260709 13:12:16.830325   703 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:17.042621   721 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:16.838522 29083 rpcz_store.cc:276] 0709 13:12:15.383214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.813399 (+1430185us) service_pool.cc:224] Handling call
0709 13:12:16.813639 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.814071 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.814098 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.814139 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:16.814166 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.814215 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.814305 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.265610   705 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:17.833737 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1992) took 2449 ms. Trace:
W20260709 13:12:17.230264 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.830439   718 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:17.835554 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.876904 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:17.837050 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1704) took 1005 ms. Trace:
I20260709 13:12:17.837451 32603 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:17.100387   662 log.cc:927] Time spent T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 2.799s	user 0.021s	sys 0.010s
I20260709 13:12:16.842898 32696 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:17.410038   679 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:17.842931 32696 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:17.847976   703 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:17.830082 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2211) took 1002 ms. Trace:
I20260709 13:12:17.851325 29008 rpcz_store.cc:276] 0709 13:12:16.827590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.832789 (+  5199us) service_pool.cc:224] Handling call
0709 13:12:16.834342 (+  1553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.834948 (+   606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.830016 (+995068us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:17.852905 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 250769us
I20260709 13:12:17.853263 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:17.839145 29151 raft_consensus.cc:2410] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:16.940572 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:17.839820 29708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.018s	sys 0.042s
W20260709 13:12:17.861164 29708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.018s	sys 0.042s
I20260709 13:12:17.839531 29077 rpcz_store.cc:276] 0709 13:12:15.383687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.834722 (+1451035us) service_pool.cc:224] Handling call
0709 13:12:16.838628 (+  3906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.838791 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.838827 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.838866 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:16.838895 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.838943 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.839030 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:16.830981   540 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:17.840322 32568 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:17.833367   705 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:17.859916 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1705) took 1021 ms. Trace:
I20260709 13:12:16.834529   678 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:17.837463 29155 rpcz_store.cc:276] 0709 13:12:16.831421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.831986 (+   565us) service_pool.cc:224] Handling call
0709 13:12:17.836991 (+1005005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.834180 29083 rpcz_store.cc:276] 0709 13:12:15.383907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.832925 (+2449018us) service_pool.cc:224] Handling call
0709 13:12:17.833240 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.833423 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.833450 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:17.833497 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:17.833531 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.833585 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:17.833684 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.913798 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1970) took 4239 ms. Trace:
W20260709 13:12:17.902393 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 48265.1us
I20260709 13:12:17.914829 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:17.906708 29151 rpcz_store.cc:276] 0709 13:12:16.838138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.838352 (+   214us) service_pool.cc:224] Handling call
0709 13:12:17.859829 (+1021477us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.834576 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 498) took 1007 ms. Trace:
I20260709 13:12:16.849220   464 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:17.916352 29713 rpcz_store.cc:276] 0709 13:12:16.826886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.834280 (+  7394us) service_pool.cc:224] Handling call
0709 13:12:17.834485 (+1000205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.333523   692 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:16.979862   441 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:17.852807   728 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:17.932075 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:17.914423 29085 rpcz_store.cc:276] 0709 13:12:13.667303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.671439 (+  4136us) service_pool.cc:224] Handling call
0709 13:12:13.671734 (+   295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:13.832715 (+160981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.832761 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:13.836195 (+  3434us) log.cc:844] Serialized 52 byte log entry
0709 13:12:14.008465 (+172270us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:14.008510 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.710072 (+2701562us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400337054. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:16.710118 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:17.099653 (+389535us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:17.906669 (+807016us) raft_consensus.cc:1685] finished
0709 13:12:17.906689 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:17.906836 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49017,"raft.queue_time_us":13588,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":361001088,"thread_start_us":171869,"threads_started":1,"wal-append.queue_time_us":364443}
W20260709 13:12:17.935338 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 72372.8us
I20260709 13:12:17.953580 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:17.903190 32568 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:17.956238 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:17.959046 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:17.959806 29151 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:17.959897 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 801) took 1141 ms. Trace:
I20260709 13:12:17.960562 29708 rpcz_store.cc:276] 0709 13:12:16.818378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.823594 (+  5216us) service_pool.cc:224] Handling call
0709 13:12:16.823905 (+   311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:16.824284 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.824313 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:16.824903 (+   590us) log.cc:844] Serialized 52 byte log entry
0709 13:12:17.861667 (+1036764us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:17.861724 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.861813 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:17.959652 (+ 97839us) raft_consensus.cc:1685] finished
0709 13:12:17.959671 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:17.959836 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":1036359,"threads_started":1,"wal-append.queue_time_us":1132620}
I20260709 13:12:17.974030 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:17.975409 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:17.955171   727 consensus_meta.cc:287] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.184s	user 0.006s	sys 0.006s
W20260709 13:12:17.933633   564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.005s	sys 0.013s
W20260709 13:12:17.994668 29050 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.008s	sys 0.009s
W20260709 13:12:17.995095 29050 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.008s	sys 0.009s
W20260709 13:12:17.995517 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 773007us
I20260709 13:12:17.997766 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:17.985591 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:17.994976   564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.005s	sys 0.013s
I20260709 13:12:17.935465   692 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:17.994109 32568 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.037081 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.035739 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.045102 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.047601 29151 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:18.046224 29159 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:17.993795   727 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.054215 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:18.053076   470 consensus_meta.cc:287] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.390s	user 0.009s	sys 0.005s
I20260709 13:12:18.082500 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.066941 32568 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:18.110440   749 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:12:18.116315   705 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.121434   727 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.123157   692 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.067960   470 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.138146 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.139364   470 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.140295   703 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:18.176014 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.176875 29151 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:18.178027 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.179220 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.180334 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.346226 29003 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:18.346421 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 680) took 3018 ms. Trace:
I20260709 13:12:18.357036 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:18.360404 29009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2415) took 2911 ms (client timeout 1947 ms). Trace:
I20260709 13:12:18.362957 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 668) took 4137 ms. Trace:
I20260709 13:12:18.365206 29076 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:18.157465 29159 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
W20260709 13:12:18.233130 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 98061.7us
I20260709 13:12:18.444450 29010 rpcz_store.cc:276] 0709 13:12:14.226699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.227019 (+   320us) service_pool.cc:224] Handling call
0709 13:12:14.252925 (+ 25906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.250072 (+997147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.250117 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.250176 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:15.250234 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.362360 (+3112126us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54001d79d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:18.362448 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.362762 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815945472}
I20260709 13:12:18.449126 29008 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:12:18.450893 29008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2220) took 1394 ms (client timeout 1696 ms). Trace:
I20260709 13:12:18.451108 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.445972 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:18.451498 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:18.452118 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1709) took 1362 ms. Trace:
I20260709 13:12:18.413720 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.452589 29158 rpcz_store.cc:276] 0709 13:12:17.090207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.090339 (+   132us) service_pool.cc:224] Handling call
0709 13:12:18.415638 (+1325299us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:18.452021 (+ 36383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25059200}
I20260709 13:12:18.453038 29006 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:18.453593 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2219) took 1478 ms. Trace:
I20260709 13:12:18.418376   634 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.454162 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.456146 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.444628 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2215) took 1604 ms. Trace:
I20260709 13:12:18.452118 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2221) took 1118 ms. Trace:
I20260709 13:12:18.457216 29010 rpcz_store.cc:276] 0709 13:12:17.334495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.450721 (+1116226us) service_pool.cc:224] Handling call
0709 13:12:18.452012 (+  1291us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:18.142150   523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.000s	sys 0.009s
W20260709 13:12:18.458843   523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.000s	sys 0.010s
I20260709 13:12:18.454005 29006 rpcz_store.cc:276] 0709 13:12:16.975319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.413127 (+1437808us) service_pool.cc:224] Handling call
0709 13:12:18.453493 (+ 40366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.459084 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:18.446641 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:18.457396 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 52644us
I20260709 13:12:18.460493 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:18.460631 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:18.460748 28976 connection.cc:256] RPC call timeout handler was delayed by -1.0738003739999158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
W20260709 13:12:18.461406 28976 connection.cc:256] RPC call timeout handler was delayed by -1.0738003739999158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
I20260709 13:12:18.456823 29003 rpcz_store.cc:276] 0709 13:12:16.836544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.836805 (+   261us) service_pool.cc:224] Handling call
0709 13:12:18.345629 (+1508824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.440708 (+ 95079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.441030 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.462680 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.463492 29003 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:18.464636 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.465320 29003 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
W20260709 13:12:18.465786 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44662.6us
W20260709 13:12:18.451349 29008 rpcz_store.cc:269] 0709 13:12:17.056822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.356276 (+1299454us) service_pool.cc:224] Handling call
0709 13:12:18.450812 (+ 94536us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.467111 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.467263 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:18.468003 29008 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:18.461407 29006 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:18.469504 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.470346 29006 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:18.432523   670 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:12:18.460486 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.475812 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:18.444087 29009 rpcz_store.cc:269] 0709 13:12:15.449041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.460237 (+ 11196us) service_pool.cc:224] Handling call
0709 13:12:18.359768 (+2899531us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540032cdd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:18.360105 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648755584}
I20260709 13:12:18.476812 29159 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:18.491557 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:18.493674 29085 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.008s	sys 0.007s
W20260709 13:12:18.494179 29085 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.008s	sys 0.007s
I20260709 13:12:18.497320 32568 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:18.495968 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:18.497320 29083 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.005s	sys 0.016s
I20260709 13:12:18.506702 29010 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:18.507473 28976 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Timed out: RequestConsensusVote RPC to 127.27.63.66:35939 timed out after 1.507s (SENT)
I20260709 13:12:18.508030 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:18.508394 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2217) took 1648 ms. Trace:
I20260709 13:12:18.508847 29003 rpcz_store.cc:276] 0709 13:12:16.859845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.507302 (+1647457us) service_pool.cc:224] Handling call
0709 13:12:18.507760 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.508093 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.508118 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.508155 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:18.508182 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.508233 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.508329 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.510859 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2218) took 1611 ms. Trace:
I20260709 13:12:18.511222 29003 rpcz_store.cc:276] 0709 13:12:16.899317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.509916 (+1610599us) service_pool.cc:224] Handling call
0709 13:12:18.510315 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.510521 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.510545 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.510587 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.510623 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.510673 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.510782 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.513046 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.513906 29150 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:12:18.514427 29150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1708) took 1461 ms (client timeout 1577 ms). Trace:
W20260709 13:12:18.514806 29150 rpcz_store.cc:269] 0709 13:12:17.053360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.053610 (+   250us) service_pool.cc:224] Handling call
0709 13:12:18.512661 (+1459051us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:18.514373 (+  1712us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38896896}
I20260709 13:12:18.441269 29005 rpcz_store.cc:276] 0709 13:12:15.327549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.328066 (+   517us) service_pool.cc:224] Handling call
0709 13:12:15.336170 (+  8104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:15.386337 (+ 50167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.386367 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:15.386791 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:12:15.562712 (+175921us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.562769 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.345402 (+2782633us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b54003f20d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:18.345519 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.345674 (+   155us) raft_consensus.cc:1685] finished
0709 13:12:18.345708 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.346163 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1976,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":3890,"spinlock_wait_cycles":114564480,"thread_start_us":169524,"threads_started":1,"wal-append.queue_time_us":272116,"wal-append.run_cpu_time_us":12645,"wal-append.run_wall_time_us":1685743}
I20260709 13:12:18.521272 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.522464 29158 raft_consensus.cc:2410] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:12:18.496802 29009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:43948
I20260709 13:12:18.523253 29009 raft_consensus.cc:606] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:18.523640 29009 raft_consensus.cc:614] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:18.523901 29158 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:12:18.534689 29158 pending_rounds.cc:85] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 5
I20260709 13:12:18.538137 29006 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:18.524830   508 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.537134 29158 pending_rounds.cc:107] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:12:18.505595   670 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:12:18.541385 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.546883 29158 raft_consensus.cc:2889] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:18.548555 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:18.502374 32568 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.562477 29008 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:18.580304 29009 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
W20260709 13:12:18.578080 28976 leader_election.cc:336] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Timed out: RequestConsensusVote RPC to 127.27.63.66:35939 timed out after 1.704s (SENT)
W20260709 13:12:18.592203 29051 connection.cc:256] RPC call timeout handler was delayed by -1.1521501050000325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
W20260709 13:12:18.592656 29051 connection.cc:256] RPC call timeout handler was delayed by -1.0774501050000254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 13:12:18.596679 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:18.579145   508 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.603077 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.602488 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:18.607116 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 76124.3us
I20260709 13:12:18.607211 29009 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:18.608620 29151 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:18.614130 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:18.616130   727 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:18.617400 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.621714   765 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:18.622720 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 686) took 1227 ms. Trace:
I20260709 13:12:18.624603 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1711) took 1217 ms. Trace:
I20260709 13:12:18.624277 29009 rpcz_store.cc:276] 0709 13:12:17.395556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.616944 (+1221388us) service_pool.cc:224] Handling call
0709 13:12:18.622070 (+  5126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.622308 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.622342 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.622407 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.622448 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.622506 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.622637 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.625152 29155 rpcz_store.cc:276] 0709 13:12:17.364020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.066430 (+702410us) service_pool.cc:224] Handling call
0709 13:12:18.068538 (+  2108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.068761 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:18.241860 (+173099us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.452844 (+210984us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.507083 (+ 54239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.507166 (+    83us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:18.537651 (+ 30485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.548168 (+ 10517us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.581373 (+ 33205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4203,"prepare.queue_time_us":177158,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":91,"raft.queue_time_us":16343,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":344,"thread_start_us":376399,"threads_started":3,"wal-append.queue_time_us":19838}
I20260709 13:12:18.626617 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 687) took 1229 ms. Trace:
I20260709 13:12:18.627075 29009 rpcz_store.cc:276] 0709 13:12:17.397378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.625682 (+1228304us) service_pool.cc:224] Handling call
0709 13:12:18.626056 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.626323 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.626348 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.626391 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:18.626421 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.626471 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.626569 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.628713 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1996) took 2925 ms. Trace:
I20260709 13:12:18.629109 29085 rpcz_store.cc:276] 0709 13:12:15.703479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.934321 (+2230842us) service_pool.cc:224] Handling call
0709 13:12:17.934634 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.934854 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:18.494702 (+559848us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.596544 (+101842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.601041 (+  4497us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:18.613440 (+ 12399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.622963 (+  9523us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.628592 (+  5629us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":609548,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":32793,"spinlock_wait_cycles":3139328,"thread_start_us":617287,"threads_started":2,"wal-append.queue_time_us":116345}
I20260709 13:12:18.650594 29008 raft_consensus.cc:2410] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:12:18.655715 32568 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.656580 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:18.656996   670 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:12:18.658042   723 log.cc:927] Time spent T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 2.322s	user 0.014s	sys 0.023s
I20260709 13:12:18.661155 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:18.663089 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:18.664065   763 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:18.665004   763 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:18.666167 29157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1690) took 1980 ms (client timeout 2225 ms). Trace:
W20260709 13:12:18.666640 29157 rpcz_store.cc:269] 0709 13:12:16.685362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.686504 (+  1142us) service_pool.cc:224] Handling call
0709 13:12:18.665746 (+1979242us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400056254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:18.665958 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342637696}
I20260709 13:12:18.616950   727 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.670225   754 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:18.670970   754 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:18.664178   657 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:18.672544   772 log.cc:927] Time spent T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:12:18.672904   657 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.671633   754 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:18.654491 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 119529us
I20260709 13:12:18.682812 29006 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 7.
I20260709 13:12:18.691038   703 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:18.691820   703 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:18.693270   703 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.693876   657 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.653024   508 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.707239   780 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:18.707863   785 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.708226   780 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.708688   785 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.710753   780 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.711040   785 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.711959   785 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:18.706321 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 689) took 1199 ms. Trace:
I20260709 13:12:18.715292 29008 rpcz_store.cc:276] 0709 13:12:17.498302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.675733 (+1177431us) service_pool.cc:224] Handling call
0709 13:12:18.676232 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.685177 (+  8945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.685205 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.697696 (+ 12491us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:18.697734 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.697802 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.697935 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.667969   763 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.698349   727 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.746869 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 690) took 1226 ms. Trace:
I20260709 13:12:18.725368   780 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:18.747494 29003 rpcz_store.cc:276] 0709 13:12:17.501105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.654527 (+1153422us) service_pool.cc:224] Handling call
0709 13:12:18.719778 (+ 65251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.726942 (+  7164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.727000 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.727063 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:18.727107 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.727185 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.727340 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.751641 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1995) took 3201 ms. Trace:
W20260709 13:12:18.753994 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 216405us
I20260709 13:12:18.754498 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 692) took 1252 ms. Trace:
I20260709 13:12:18.755352 29008 rpcz_store.cc:276] 0709 13:12:17.501677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.719116 (+1217439us) service_pool.cc:224] Handling call
0709 13:12:18.726198 (+  7082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.752857 (+ 26659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.752899 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.754138 (+  1239us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:18.754190 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.754287 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.754428 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.712702   785 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.762540 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1993) took 3367 ms. Trace:
I20260709 13:12:18.763034 29077 rpcz_store.cc:276] 0709 13:12:15.384163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.862950 (+2478787us) service_pool.cc:224] Handling call
0709 13:12:17.902915 (+ 39965us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.903343 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.903379 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:17.903913 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:12:18.495769 (+591856us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.593412 (+ 97643us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.706179 (+112767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.718930 (+ 12751us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.720911 (+  1981us) raft_consensus.cc:1685] finished
0709 13:12:18.720929 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.751201 (+ 30272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38126,"prepare.queue_time_us":74731,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":121027,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":15338,"spinlock_wait_cycles":1098496,"thread_start_us":576503,"threads_started":2,"wal-append.queue_time_us":527770}
I20260709 13:12:18.766938   493 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.769910   682 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.772755 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1990) took 3389 ms. Trace:
I20260709 13:12:18.773136 29076 rpcz_store.cc:276] 0709 13:12:15.383461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.832100 (+1448639us) service_pool.cc:224] Handling call
0709 13:12:16.832513 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.452170 (+1619657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.772437 (+320267us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400337554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:18.772658 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3523255552}
I20260709 13:12:18.776460 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.776767 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 693) took 1274 ms. Trace:
W20260709 13:12:18.777603 29710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 502) took 1788 ms (client timeout 1599 ms). Trace:
W20260709 13:12:18.778074 29710 rpcz_store.cc:269] 0709 13:12:16.988938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.989106 (+   168us) service_pool.cc:224] Handling call
0709 13:12:18.776107 (+1787001us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:18.777452 (+  1345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15921024}
I20260709 13:12:18.779891 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:18.780802 29709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 506) took 1405 ms (client timeout 1837 ms). Trace:
I20260709 13:12:18.777354 29003 rpcz_store.cc:276] 0709 13:12:17.501963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.752537 (+1250574us) service_pool.cc:224] Handling call
0709 13:12:18.776109 (+ 23572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.776353 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.776392 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.776440 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:18.776484 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.776550 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.776674 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.782261 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 694) took 1179 ms. Trace:
I20260709 13:12:18.783051 29008 rpcz_store.cc:276] 0709 13:12:17.603011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.780212 (+1177201us) service_pool.cc:224] Handling call
0709 13:12:18.780721 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.781007 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.781974 (+   967us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.782021 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:18.782053 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.782107 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.782200 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.793430 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 695) took 1179 ms. Trace:
I20260709 13:12:18.797710 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1979) took 5021 ms. Trace:
I20260709 13:12:18.800437 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 691) took 1299 ms. Trace:
I20260709 13:12:18.791356 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2222) took 1367 ms. Trace:
I20260709 13:12:18.800941 29006 rpcz_store.cc:276] 0709 13:12:17.501404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.714257 (+1212853us) service_pool.cc:224] Handling call
0709 13:12:18.746351 (+ 32094us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.800005 (+ 53654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.800033 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.800094 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:18.800138 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.800217 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.800366 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:18.781211 29709 rpcz_store.cc:269] 0709 13:12:17.375329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.398077 (+ 22748us) service_pool.cc:224] Handling call
0709 13:12:18.779565 (+1381488us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:18.780723 (+  1158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13701888}
I20260709 13:12:18.798995 29003 rpcz_store.cc:276] 0709 13:12:17.614243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.792253 (+1178010us) service_pool.cc:224] Handling call
0709 13:12:18.792673 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.793045 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.793082 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.793125 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.793174 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.793234 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.793348 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:18.802460 29005 rpcz_store.cc:276] 0709 13:12:17.390492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.521428 (+1130936us) service_pool.cc:224] Handling call
0709 13:12:18.521805 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.522197 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:18.581994 (+ 59797us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.751408 (+169414us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.753557 (+  2149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.753622 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.753679 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.757250 (+  3571us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.758074 (+   824us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18170,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":107,"raft.queue_time_us":270,"raft.run_wall_time_us":6354,"spinlock_wait_cycles":60114176,"thread_start_us":205845,"threads_started":2,"wal-append.queue_time_us":180308}
I20260709 13:12:18.795686 32630 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:18.796139 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2213) took 1893 ms. Trace:
W20260709 13:12:18.794229 28975 leader_election.cc:336] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.806s (SENT)
I20260709 13:12:18.814119   508 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.798137 29079 rpcz_store.cc:276] 0709 13:12:13.775955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.054838 (+2278883us) service_pool.cc:224] Handling call
0709 13:12:16.055193 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:16.195226 (+140033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.195258 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:16.195675 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 13:12:16.505633 (+309958us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:16.505680 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.797014 (+2291334us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000465d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:18.797089 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.797173 (+    84us) raft_consensus.cc:1685] finished
0709 13:12:18.797191 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.797516 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321447680,"thread_start_us":170898,"threads_started":1,"wal-append.queue_time_us":327714,"wal-append.run_cpu_time_us":3558,"wal-append.run_wall_time_us":1217129}
I20260709 13:12:18.814865 32630 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.816543 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.820712 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.822208 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.823448 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.824806 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:18.827454 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.826813 32570 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:18.834776   754 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.753026 29083 rpcz_store.cc:276] 0709 13:12:15.549863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.912679 (+2362816us) service_pool.cc:224] Handling call
0709 13:12:17.913053 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.913294 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:18.524429 (+611135us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.687360 (+162931us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.714459 (+ 27099us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.720198 (+  5739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.745817 (+ 25619us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.751581 (+  5764us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3644,"prepare.queue_time_us":630767,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":109231,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"thread_start_us":708584,"threads_started":3,"wal-append.queue_time_us":108401}
I20260709 13:12:18.842057 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:18.836455 32570 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:18.843302 32570 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:18.852319 29079 raft_consensus.cc:2376] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10. Re-sending same reply.
I20260709 13:12:18.813063 29001 rpcz_store.cc:276] 0709 13:12:16.831486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.835760 (+  4274us) service_pool.cc:224] Handling call
0709 13:12:18.421979 (+1586219us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:18.422324 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.422778 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:18.479294 (+ 56516us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.689285 (+209991us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.720608 (+ 31323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.720680 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:18.720735 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.720827 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.720994 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31016,"prepare.queue_time_us":115340,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":108,"raft.queue_time_us":92385,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":27765,"spinlock_wait_cycles":17536256,"thread_start_us":201300,"threads_started":3,"wal-append.queue_time_us":81176}
I20260709 13:12:18.857112 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:18.857658   493 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:18.857988 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.858444   493 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.860458   790 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:18.860682   785 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.865127   493 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.872622 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.872958 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 558) took 3826 ms. Trace:
I20260709 13:12:18.849164 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.873504 29156 rpcz_store.cc:276] 0709 13:12:15.014282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.661649 (+647367us) service_pool.cc:224] Handling call
0709 13:12:15.670890 (+  9241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.671088 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:15.714670 (+ 43582us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.705200 (+2990530us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b180037c7e0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:12:18.791207 (+ 86007us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.793085 (+  1878us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:18.794358 (+  1273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.798130 (+  3772us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.840639 (+ 42509us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":43532,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":6802,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":201,"spinlock_wait_cycles":9698688,"thread_start_us":43085,"threads_started":1,"wal-append.queue_time_us":3081790}
I20260709 13:12:18.848287   779 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:18.793445   780 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.875100 29005 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:18.882607 29710 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:18.886992   787 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:18.887590   530 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:18.887717 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46236.4us
I20260709 13:12:18.888376   779 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.892195 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.893213 32570 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:18.896209 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 49708.9us
I20260709 13:12:18.896384 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.898984   754 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:18.900604   745 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 2.091s	user 0.005s	sys 0.032s
I20260709 13:12:18.901453 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:18.903487   508 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:12:18.903939 29007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33750 (request call id 683) took 3535 ms (client timeout 1639 ms). Trace:
I20260709 13:12:18.904968   727 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:18.906498 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.907222 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.907373 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.909860 29076 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:12:18.910245 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.910452   777 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.911586   785 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:18.911739 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1520) took 4662 ms. Trace:
I20260709 13:12:18.912154 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:18.912607   787 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.900627   453 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:18.929379 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:18.913429   780 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.913699   725 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.913659 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:18.914736 29048 leader_election.cc:336] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.865s (SENT)
I20260709 13:12:18.920301 29005 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:18.934623 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:18.924080 29076 pending_rounds.cc:85] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 4
I20260709 13:12:18.935382 29076 pending_rounds.cc:107] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:12:18.935204 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.935649 29076 raft_consensus.cc:2889] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:18.937021 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.925483 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.926468 29082 rpcz_store.cc:276] 0709 13:12:14.248947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.561988 (+2313041us) service_pool.cc:224] Handling call
0709 13:12:16.562362 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.696890 (+134528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.911364 (+2214474us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540034dd54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:18.911585 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308109568}
W20260709 13:12:18.923111 29007 rpcz_store.cc:269] 0709 13:12:15.368774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.903600 (+3534826us) service_pool.cc:211] Skipping call since client already timed out
0709 13:12:18.903749 (+   149us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:12:18.935210 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:18.926645 29710 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:18.929196 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.930966 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 1987) took 5044 ms. Trace:
I20260709 13:12:18.939990 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:18.940944 29683 connection.cc:256] RPC call timeout handler was delayed by -2.25148346900005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:12:18.941390 29683 connection.cc:256] RPC call timeout handler was delayed by -1.9382770330000767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
W20260709 13:12:18.941769 29683 connection.cc:256] RPC call timeout handler was delayed by -1.8041770330000872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
I20260709 13:12:18.930755   727 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:18.942193 29683 connection.cc:256] RPC call timeout handler was delayed by -1.1678423399999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 13:12:18.942569 29683 connection.cc:256] RPC call timeout handler was delayed by -1.158842340000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
I20260709 13:12:18.942901 29079 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:18.931429 32570 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.940079 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.943537 29077 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:18.943703   727 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.923753 29083 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:18.942577   755 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:18.944455 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:18.944796 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:18.944558   755 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:18.937605 29076 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:18.938059 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:18.942873 29683 connection.cc:256] RPC call timeout handler was delayed by -1.1327423400000498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260709 13:12:18.946942 29683 connection.cc:256] RPC call timeout handler was delayed by -1.1102423400000134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
I20260709 13:12:18.943781 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.945700 29083 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:18.939930 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.948096 29680 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:18.950907 29048 leader_election.cc:336] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.947s (SENT)
I20260709 13:12:18.951205 29710 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:12:18.949671 29079 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:18.955802 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.956246 29005 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:18.939373 29080 rpcz_store.cc:276] 0709 13:12:13.886444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.498857 (+2612413us) service_pool.cc:224] Handling call
0709 13:12:16.507645 (+  8788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:16.596660 (+ 89015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.596696 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:16.644530 (+ 47834us) log.cc:844] Serialized 26 byte log entry
0709 13:12:16.809084 (+164554us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:16.809123 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.809188 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.875368 (+2066180us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:18.930494 (+ 55126us) raft_consensus.cc:1685] finished
0709 13:12:18.930519 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.930806 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":530,"raft.queue_time_us":13290,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":36369,"spinlock_wait_cycles":52409600,"thread_start_us":194991,"threads_started":2,"wal-append.queue_time_us":120908}
I20260709 13:12:18.957651 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.958336 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.958503 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.958912 29080 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:18.959910 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.960355 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.960934 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.959811 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.961499   779 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:18.965301 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:18.966760 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.967525 32570 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:18.968149 32570 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:18.969591   519 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:18.970309 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 664) took 4716 ms. Trace:
I20260709 13:12:18.970249   519 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.970644 32570 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:18.929453   785 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:18.970993 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.013459 29708 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:19.013674   787 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:19.014083 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.015197 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.016121   785 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.012825 29713 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
W20260709 13:12:19.016783 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.970800 29081 rpcz_store.cc:276] 0709 13:12:14.253559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.562954 (+2309395us) service_pool.cc:224] Handling call
0709 13:12:16.583333 (+ 20379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.738956 (+155623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.969849 (+2230893us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400336b54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:18.970142 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264135040}
I20260709 13:12:19.017915 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.018668 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:19.018735 29715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 503) took 1989 ms (client timeout 2019 ms). Trace:
W20260709 13:12:19.020205 29715 rpcz_store.cc:269] 0709 13:12:17.028961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.029451 (+   490us) service_pool.cc:224] Handling call
0709 13:12:19.017626 (+1988175us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:19.018676 (+  1050us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14852864}
I20260709 13:12:19.018324 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.021785   635 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.022063   783 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:19.022269 29081 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:19.023536 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.024210 29080 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:19.024845   783 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:19.025216 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:18.947839   755 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:19.016319 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.026299 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:18.963898 29048 leader_election.cc:336] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.948s (SENT)
W20260709 13:12:19.017170 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45743.1us
W20260709 13:12:19.028110 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 53384.1us
W20260709 13:12:18.971467 29125 connection.cc:256] RPC call timeout handler was delayed by -2.0444378500000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
W20260709 13:12:19.034500 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 63953.4us
I20260709 13:12:19.036394 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.036881 29081 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:19.037935 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:18.963528   754 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:19.032618 29051 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.040004 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:19.043259 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 133600us
I20260709 13:12:19.047254 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:19.048521   793 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:19.013675   519 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:19.052788 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 560) took 1068 ms. Trace:
I20260709 13:12:19.053177 29159 rpcz_store.cc:276] 0709 13:12:17.984710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.612318 (+627608us) service_pool.cc:224] Handling call
0709 13:12:18.676475 (+ 64157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.695713 (+ 19238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.695745 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.695823 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:18.695889 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.695974 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.052611 (+356637us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b5400056250. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:19.052731 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37186432}
I20260709 13:12:18.971931   725 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:19.104185 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.131331 29151 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:19.104857 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.107236 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.113750 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:19.124235 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 86372.3us
I20260709 13:12:19.123674 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.105334 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.129102 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:19.134927 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 157628us
I20260709 13:12:19.105374 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.139098 29079 raft_consensus.cc:2393] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:19.138789 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 100729us
I20260709 13:12:19.133590 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.141909 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.140646 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:19.135293 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 90544us
I20260709 13:12:19.144554 29159 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:19.144093 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:18.953321   777 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.149302 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.149977 29714 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.152078 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.153324 29081 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 5.
I20260709 13:12:19.138978 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.155561 29005 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:19.156118 29080 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:19.124938   754 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:19.129511   793 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:19.152732 29150 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
W20260709 13:12:19.158547   746 log.cc:927] Time spent T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 2.348s	user 0.019s	sys 0.030s
I20260709 13:12:19.161295 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:19.162149 29154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1707) took 2139 ms (client timeout 2162 ms). Trace:
I20260709 13:12:19.162580 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.163807 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:19.164198 29005 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:19.164297 32570 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:19.206889 29714 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:19.165187 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:19.170178 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 218588us
I20260709 13:12:19.156664 29082 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:19.177260   795 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:19.177927 29711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 505) took 1954 ms (client timeout 1659 ms). Trace:
I20260709 13:12:19.142017   635 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:19.184712 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.208194 29081 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:12:19.207835 29711 rpcz_store.cc:269] 0709 13:12:17.223507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.223639 (+   132us) service_pool.cc:224] Handling call
0709 13:12:19.106805 (+1883166us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:19.177866 (+ 71061us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17883008}
I20260709 13:12:19.208734   667 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:19.209206   667 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:19.209259 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:19.209718   667 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.210680 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.210964 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.211321 29081 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:19.211660 29078 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:19.212447 29153 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:19.212574 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.213728 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:19.201354 29154 rpcz_store.cc:269] 0709 13:12:17.023062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.023224 (+   162us) service_pool.cc:224] Handling call
0709 13:12:19.051944 (+2028720us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:19.162066 (+110122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60284672}
I20260709 13:12:19.214596 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.214989 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.215169 29152 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
W20260709 13:12:19.215723 29152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1710) took 1949 ms (client timeout 1806 ms). Trace:
I20260709 13:12:19.216341 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.216646 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:19.216941 29154 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:19.217633 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:19.217937 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.218552   704 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:19.218950   704 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:19.219060 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.219358   704 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.220750   765 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:19.221177   765 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:19.221385   790 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:19.221556   765 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:19.216114 29152 rpcz_store.cc:269] 0709 13:12:17.266843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.267036 (+   193us) service_pool.cc:224] Handling call
0709 13:12:19.214299 (+1947263us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b5800003398. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:19.215639 (+  1340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39433984}
I20260709 13:12:19.225489 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.225620 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.226127 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.226584 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.226708 29077 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:19.227632 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.228797 32570 raft_consensus.cc:3055] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:12:19.229205 29717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 504) took 2134 ms (client timeout 2157 ms). Trace:
I20260709 13:12:19.229542 32570 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:19.229662 29717 rpcz_store.cc:269] 0709 13:12:17.096814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.096936 (+   122us) service_pool.cc:224] Handling call
0709 13:12:19.050911 (+1953975us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b58001a1698. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:19.229134 (+178223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18850688}
I20260709 13:12:19.230484 32570 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:19.190315 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 152877us
I20260709 13:12:19.231743 32570 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.196338 29151 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:19.185765 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.236430 29085 raft_consensus.cc:2393] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:19.230887 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.232009 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:19.157085 29051 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.232544   470 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.173375   754 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.226596 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.233287 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.234710 29078 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
W20260709 13:12:19.233103 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 69395.2us
I20260709 13:12:19.233718 29077 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:19.236429 29079 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:19.238716 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.207099 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.240319 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.227734 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.189782 29083 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:19.241631 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2216) took 2403 ms. Trace:
I20260709 13:12:19.237718   519 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:19.242748 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.243372 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:19.241082 29152 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:19.242643   519 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:19.245698 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.246469 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.247684 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.242321 29002 rpcz_store.cc:276] 0709 13:12:16.838271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.839525 (+  1254us) service_pool.cc:224] Handling call
0709 13:12:19.055190 (+2215665us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a8070a0 00007f411a806984 00007f4113d816d4 00007f4113d81662
0709 13:12:19.241553 (+186363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149833472}
W20260709 13:12:19.249518 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44155.2us
I20260709 13:12:19.251451 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.251739 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.252857 29081 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:19.252442 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.255048 29079 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:19.256695 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.254449 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.257972 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.258184 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:19.258280 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 111202us
I20260709 13:12:19.259053 29077 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:19.264183   635 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:19.265745 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:19.266734   564 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:19.267657   564 consensus_queue.cc:1243] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer ca45420e033847489ad989c1c6eed3ef log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:12:19.267810 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.261813   453 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:12:19.273550 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.274748 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.275231 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.275853 29077 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:19.276384 29714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 499) took 2442 ms. Trace:
I20260709 13:12:19.276684 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:19.277825 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.278409 29077 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:19.279196 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:19.279693 29077 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:19.280296 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.281092 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.281373 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.281787 29082 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:19.282392 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.268316   669 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:19.283017   669 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:19.283406   669 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:19.250490   519 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.264241 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.262226 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.268193 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 688) took 1770 ms. Trace:
I20260709 13:12:19.282477 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.288249 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.288365 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.276722 29714 rpcz_store.cc:276] 0709 13:12:16.833546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.833669 (+   123us) service_pool.cc:224] Handling call
0709 13:12:16.833921 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:19.232183 (+2398262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.276120 (+ 43937us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b54000154d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:19.276307 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5245621632}
I20260709 13:12:19.284890 29079 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:19.275254 29080 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:19.289872 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.288458 29082 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:19.292737 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.287974 29009 rpcz_store.cc:276] 0709 13:12:17.497566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.629089 (+1131523us) service_pool.cc:224] Handling call
0709 13:12:18.629468 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.938031 (+308563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.938058 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.938094 (+    36us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:18.938122 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.267932 (+329810us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54001caf54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:19.267985 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.268120 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709132160}
I20260709 13:12:19.295909 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.297297 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.295961 29080 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
W20260709 13:12:19.303522 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 93571.3us
I20260709 13:12:19.305496 29083 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:19.258598   782 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:19.310657 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.311208 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.307847 29150 raft_consensus.cc:2410] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:12:19.308485 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61930.1us
I20260709 13:12:19.309214 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.245815   470 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.319972 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:19.318150 28976 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.66:35939 timed out after 2.903s (SENT) [suppressed 67 similar messages]
W20260709 13:12:19.318943 28892 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 06ae4197fd784525b695a9b16c8094ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:19.323302 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 666) took 5041 ms. Trace:
I20260709 13:12:19.323743 29084 rpcz_store.cc:276] 0709 13:12:14.282219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.690743 (+2408524us) service_pool.cc:224] Handling call
0709 13:12:16.691042 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:16.691373 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.691400 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:16.699118 (+  7718us) log.cc:844] Serialized 52 byte log entry
0709 13:12:16.834947 (+135829us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:16.922420 (+ 87473us) log.cc:844] Serialized 14 byte log entry
0709 13:12:16.922681 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.922777 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:17.362330 (+439553us) spinlock_profiling.cc:243] Waited 9.65 ms on lock 0x7b1800260fe0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:12:19.146343 (+1784013us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:19.322922 (+176579us) raft_consensus.cc:1685] finished
0709 13:12:19.322945 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.323126 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":96013,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":46,"raft.queue_time_us":835,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":6295,"spinlock_wait_cycles":75744896,"thread_start_us":77186,"threads_started":1,"wal-append.queue_time_us":68431}
I20260709 13:12:19.325233   636 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.326082 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.326246 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.326867 29084 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:19.327224 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.327791 29081 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:12:19.337556   792 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:12:19.310379   782 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.244102   667 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.338820 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.339973 29077 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:19.342471 29083 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:19.342990 29158 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:19.345427 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.347191 29159 raft_consensus.cc:2410] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:12:19.348351 29080 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:19.349367 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.349541 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.350066 29079 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:19.351754 29051 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.349928 29076 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:19.354233 29158 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:19.355734 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.357861 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.358042 32630 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:19.358470 29076 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:19.359301 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.359700   683 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:19.360186   683 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:19.359791 29076 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:19.362726 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.364526 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.361120   683 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.365586   777 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:19.349263   794 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:12:19.365674   683 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.364570 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.368417 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.368706 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:19.369349 29083 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 7.
I20260709 13:12:19.337697   793 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.343859   782 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.374434 29150 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:19.400851 29080 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:19.401787 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.402388 29010 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:19.372737 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:19.375607 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 59837.7us
W20260709 13:12:19.377444 29153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1697) took 2607 ms (client timeout 1627 ms). Trace:
I20260709 13:12:19.377959 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.382493 29157 raft_consensus.cc:2410] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:19.367805   667 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:19.385206 29078 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:19.416891 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.385408 29152 raft_consensus.cc:2410] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:12:19.386022 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:19.387358   786 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:19.389462   470 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.387156 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.391554 29081 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:19.426530 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 53895.1us
I20260709 13:12:19.395926 29083 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:12:19.405187 29153 rpcz_store.cc:269] 0709 13:12:16.770469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.770704 (+   235us) service_pool.cc:224] Handling call
0709 13:12:19.375774 (+2605070us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b54002a2bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:19.377383 (+  1609us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5572690688}
I20260709 13:12:19.412675 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2207) took 3713 ms. Trace:
I20260709 13:12:19.405494 29009 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:19.418957 29156 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:19.403429   777 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:19.425336 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:19.427063 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 80697.9us
I20260709 13:12:19.434795 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.437120 29001 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:19.431761 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.443707 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.444332 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.444849 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.446722   667 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:19.447383   519 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.450121   519 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:19.452512 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.453675 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.454361 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:19.455032 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.455255   636 consensus_queue.cc:1059] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:19.455967   787 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:19.456429   787 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:19.456750 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.457527   669 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.458072 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.459363 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.459681 29051 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.417387   765 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.460683 29156 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:19.460568   705 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:19.457885   470 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:19.462003   765 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.462296   705 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:19.462591   765 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:19.453689 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.455783 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.456939   787 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:19.463605 29158 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:19.455328 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.465626 29150 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:19.467936 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.445343 29010 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:19.455775 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:19.472077 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.472226 29122 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.473088 29122 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:19.473595   530 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:19.474040   530 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:19.475158   519 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:19.475044   530 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.475881   519 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:19.476270   519 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.476680   519 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:19.477121 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.477946 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:19.478565 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:19.479413   686 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:19.479506 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.479866   686 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:19.480311   686 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:19.480525   669 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:19.484228   635 raft_consensus.cc:3055] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:19.484642   635 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:19.485570   635 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.486912   635 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:19.473975 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.430475 29004 rpcz_store.cc:276] 0709 13:12:15.699313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.330135 (+630822us) service_pool.cc:224] Handling call
0709 13:12:16.330525 (+   390us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:18.892967 (+2562442us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b180036d000. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f4115839060 00007f4115839bfb 00007f411584464b
0709 13:12:18.893179 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.893205 (+    26us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:12:18.893918 (+   713us) log.cc:844] Serialized 78 byte log entry
0709 13:12:18.948566 (+ 54648us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:18.954427 (+  5861us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.963014 (+  8587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.026777 (+ 63763us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:19.412277 (+385500us) raft_consensus.cc:1685] finished
0709 13:12:19.412380 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.412517 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5746,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":42299,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":78938,"spinlock_wait_cycles":171426688,"thread_start_us":90344,"threads_started":2,"wal-append.queue_time_us":248905}
I20260709 13:12:19.463027   765 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.495270   765 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:19.496977 32570 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:19.498137   667 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:19.498493   667 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:19.499356   667 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.502246   795 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:19.493541   669 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:19.502988 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 47290.8us
I20260709 13:12:19.504215   530 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:19.476176   692 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:19.500195   780 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:19.508394 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.509742   658 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:12:19.509163 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 122091us
I20260709 13:12:19.439383   754 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.518559 28975 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.513972   520 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:12:19.536150 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56733.6us
I20260709 13:12:19.537263 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.514143   705 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:19.514878   795 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:19.553470 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:19.541496   755 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.504s	user 0.005s	sys 0.005s
I20260709 13:12:19.554263   780 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:19.555226   755 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.557614 29156 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:19.558238 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:19.558773 29010 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.559646 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.529021   658 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:19.514955   683 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
W20260709 13:12:19.582289 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 122730us
I20260709 13:12:19.540376   520 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:19.541353   705 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:19.527292   686 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:19.591362 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 100927us
W20260709 13:12:19.592974 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 74481.9us
I20260709 13:12:19.587431 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:19.593967 28975 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.594877 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.596451 29002 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:19.599407 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.600010 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.581877   692 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:19.602842 29151 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:19.605221   669 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:19.560577   795 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:19.580717   658 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:19.611605 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:19.614677 29714 raft_consensus.cc:2393] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:19.630909   777 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:19.630744   795 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.631475   777 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.631958   777 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.583771   667 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.633565 32630 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:19.633960 32630 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:19.634954 32630 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.636322   780 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:19.635950   795 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.648572 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.649893   795 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:19.651546 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 123675us
I20260709 13:12:19.654011 29683 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.650300 29050 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.655588 29051 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:19.657804 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:19.658774 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 47875.3us
I20260709 13:12:19.606994   705 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:19.664624 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 44949.5us
I20260709 13:12:19.667456 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.620149   530 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:19.669358 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.672770   703 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:19.591145   790 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:12:19.656739   636 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:19.674511 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 78328.2us
I20260709 13:12:19.676684 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:19.676167 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 73650.3us
W20260709 13:12:19.679177 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 142024us
I20260709 13:12:19.681183 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.673209   703 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.682662 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.619968   704 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.684444 29683 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.671635   780 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:19.673866   725 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:19.677567   636 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.718456   782 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.721390 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.723464 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.675491   576 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:12:19.726009 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.688162   530 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:19.749948 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 82618.7us
I20260709 13:12:19.759312 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.759140   658 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.755834   782 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:19.762034 29152 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:12:19.763099 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.764345   635 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:19.765076   635 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:19.765863 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:19.766686   635 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:19.767056   635 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:19.767416   635 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:19.773411 29050 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:19.734894   519 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.755478   686 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.688180   780 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.736446 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.757377   704 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:19.724502   636 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:19.752281   537 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:19.789768 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.789867 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.791112   811 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:19.772964 28906 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 5 to 7, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 7 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:19.774319   754 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.776793   635 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:19.778841 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53038.9us
I20260709 13:12:19.756906 32630 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.685429   703 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.777177   765 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.810889 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.787422 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.787516 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:19.734337   795 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:19.796737 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:19.784207 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 105894us
I20260709 13:12:19.736281   786 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:19.735759   755 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.798807   635 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.799631   777 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.801857   519 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:19.735497 29001 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:19.803437   658 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.825376   635 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:19.806754 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.807785 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.808668 29009 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:19.808807 29004 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:12:19.734958 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 50660.8us
I20260709 13:12:19.809327 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.810889 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.797923 29152 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:19.828178 29008 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:19.817903 29150 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:19.783368   780 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.724689   520 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.821092 29006 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:19.817548   692 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:19.833433 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.838913 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:19.841033 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:19.846550   816 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:19.846992   816 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:19.847337   816 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:19.818857   795 raft_consensus.cc:2764] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:12:19.857041 29010 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:19.848171   470 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:19.874727 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 78037.2us
I20260709 13:12:19.874904   576 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:19.875509 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:19.876425 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.877203   754 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:19.877606   754 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:19.878587   754 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 8.11, Last appended by leader: 11, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.880766 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.881145 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:19.881734 29003 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.883028   800 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:19.884545 29005 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:19.885710 29005 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:12:19.888337 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:19.889680   686 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:19.890080   686 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:19.890430   686 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.892441   530 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:19.893354 29076 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:19.898134 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:19.898468 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.877648 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:19.900173 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:19.900269 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.902021 29005 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:19.904996 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:19.909070 29003 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
W20260709 13:12:19.909435 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 239234us
I20260709 13:12:19.904570 28906 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:19.903138 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.910991 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.915567 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:19.916128 29006 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:19.943233 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 65649.4us
I20260709 13:12:19.912518   530 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:19.950160   777 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.951179   787 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.899147   786 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:12:19.953366   765 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:19.954743 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:19.938501   810 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:19.870047   755 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:12:19.956671   636 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.958923   816 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.961158   679 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:19.963717   520 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:19.964165   692 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:19.948740 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.973726 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.980981 29006 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
W20260709 13:12:20.006506 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 219148us
W20260709 13:12:20.016520 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 337872us
I20260709 13:12:20.107489 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:20.110011 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 167070us
I20260709 13:12:20.114056 29007 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:20.113323 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 164632us
I20260709 13:12:20.118983 29122 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:19.982271   530 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:20.124354 29051 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:19.969161   780 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:19.981603   705 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.142555 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:20.128695   683 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:20.130350   777 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:20.151008   777 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.143745   519 raft_consensus.cc:2764] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:12:20.172670   530 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.142639   795 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:12:20.231166   777 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:20.236382   636 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:20.236137   686 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.238315   686 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:20.239146   686 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:20.239506   686 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:20.239862   686 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:20.241286 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 111999us
W20260709 13:12:20.258077 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 310003us
I20260709 13:12:20.259052 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:20.221205   782 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:20.269876 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:20.274175 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:20.150234   703 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.283504 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:20.286049 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:20.284642 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 160343us
I20260709 13:12:20.276556   537 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:12:20.294286   705 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:20.280527   795 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:20.268296   635 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:12:20.280539   530 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:20.298035 32570 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:20.305600   817 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:20.306087   822 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:12:20.306674 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:20.307665   780 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:20.283339   683 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:20.295415 32630 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:20.345173 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52754.8us
I20260709 13:12:20.344353   837 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:20.340437   705 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:20.339661 32570 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:20.348857 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:20.285523   782 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 13:12:20.343968 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 208512us
I20260709 13:12:20.343083   817 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.356465 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:20.350203   837 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:20.358312 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:20.348158   683 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:20.364348 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:20.350101 32630 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:20.367661 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:20.353060   705 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.350379   787 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:20.342590   790 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:12:20.392184 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:20.386351 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 79766.3us
W20260709 13:12:20.369596 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.363348   817 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:20.393692 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:20.404228   825 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:12:20.406670   829 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:20.407195   829 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.407644   829 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:20.384667   683 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:20.409803 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61025.3us
I20260709 13:12:20.389334   537 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:20.391136   705 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.415313 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:20.420221 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 158129us
W20260709 13:12:20.423086 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.428797 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:20.430550 29007 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:20.413132   822 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:20.414723   811 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:20.454806 29001 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:20.457211 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:20.459404 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 47002.8us
I20260709 13:12:20.416844   537 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:20.461622 29125 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:20.465994 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:20.466946   470 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:20.408329 32630 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.468218   826 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:20.468616   826 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:20.469415   826 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.471663   816 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:20.452381   822 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:20.473306   823 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:12:20.456782   811 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:20.464231   537 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:20.475720   564 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:20.553406 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 76729.4us
I20260709 13:12:20.558528 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:20.475263   835 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:20.559484 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 102355us
I20260709 13:12:20.507778   811 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:20.568143   782 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:20.584197   824 log.cc:927] Time spent T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.277s	user 0.001s	sys 0.001s
I20260709 13:12:20.584435   782 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:20.584900   782 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:20.588609   851 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:20.588936   843 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.138s	user 0.002s	sys 0.000s
W20260709 13:12:20.589713   841 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:12:20.590276   813 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:20.591049   832 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:20.592685   834 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:12:20.595160   837 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.596347   822 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.583387   686 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.597419   822 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:20.598479 29003 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:20.600499   840 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:12:20.600919 29083 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:20.601522 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 141167us
I20260709 13:12:20.601673   686 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:20.601940 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:20.602231 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:20.602720 29077 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 6: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:20.603205   811 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.603440 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:20.604590   754 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.604787   851 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:20.605249   851 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.605599   851 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:20.611337 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:20.617452   686 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:20.618242   686 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:20.618867   777 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.619343   686 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.620332   564 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:20.623550   669 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:20.623978   669 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:20.624440   669 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:20.624670   811 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:20.602720   782 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.627568   782 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:20.604616 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:20.565083   835 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.636693   831 consensus_queue.cc:1059] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:20.638690   835 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:20.638684   844 raft_consensus.cc:3055] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:20.643889   844 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:20.644452   520 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.000s
I20260709 13:12:20.644878   844 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.649019 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:20.650138 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:20.651585   782 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:20.651979   782 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:20.652396   837 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:20.652508   782 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:20.653231   679 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:20.655411   519 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:20.655772   519 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:20.656829   519 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.657946   519 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:20.660432 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:20.661021 29158 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:20.665203 32570 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.666222 29157 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:20.669035   780 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:20.695778   780 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:20.466985 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:20.696936   780 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.697908   780 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:20.652055   814 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:20.699815   814 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:20.700227   814 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:20.700277   826 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:20.681452   777 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:20.685266   703 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:20.688059 29122 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:20.708318   783 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:20.708825   783 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:20.709695   783 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.711089   679 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:20.712199   520 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:20.676977 28906 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:20.712749   520 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:20.712127   679 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:20.714648   679 consensus_queue.cc:1243] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:20.713579   520 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.695186   828 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:20.716966   828 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.717504   828 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:20.746644   782 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.746930   829 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.748668 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:20.749002   782 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:20.750218 29151 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:20.751349 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 146793us
I20260709 13:12:20.752194   777 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.714128 28906 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 3 to 8, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:20.752842   777 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.754014 28906 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 3 to 6, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 6 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:20.755462   777 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:20.756181 32570 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:20.761663   810 consensus_meta.cc:287] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.774s	user 0.009s	sys 0.003s
I20260709 13:12:20.762113   810 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:20.763759 29122 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:20.765471   782 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:20.766661   530 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.767180   530 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.768383   520 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:20.769991   530 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:20.771538 29080 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:20.772716 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:20.773898 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:20.777402 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:20.777621   683 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.752156 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:20.781675   683 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:20.784478   849 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:20.785136   849 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.785637   849 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:20.815375 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.828584 29151 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:20.831375   829 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:20.847324 29001 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:20.857020 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 104938us
I20260709 13:12:20.857859 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:20.859040 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:20.859771   530 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:20.860368   530 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:20.860807   530 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:20.863595 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 400245us
I20260709 13:12:20.864657 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:20.865049   844 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:20.866591   852 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:20.867417   852 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:20.877372 29007 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:12:20.877898 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:20.879638 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:20.880903 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.884166 29714 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:12:20.884928   855 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:12:20.882946 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:20.886884 29157 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:20.888309 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:20.755692 28906 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 4 to 9, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 9 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:20.898108 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:20.899812 29004 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:20.899897 29151 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:20.892396 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:20.870656   829 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:20.910578 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:20.911429 29078 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:20.870617   683 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:20.918382 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:20.915941 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:20.909515   810 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:20.922313 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:20.929790 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:20.929800 29715 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:20.924217 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:20.922803 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 44970.7us
I20260709 13:12:20.931728 29151 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:20.932610 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:20.933462 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:20.934388 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:20.936836 29711 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:20.879747   576 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:12:20.944115 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:20.945401 29005 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:20.940526 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:20.940455 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:20.946954 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:20.910158   829 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.940438 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:20.941525 28906 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 3 to 5, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 5 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:20.955693 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:20.963755 29082 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:20.965358 29714 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:20.966446 29714 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:20.929409   810 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:20.975001 29009 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:20.951612   829 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:20.976194 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:20.982930   669 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.974042 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:20.948194   853 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:20.917019   530 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:20.929507   523 consensus_queue.cc:1059] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
W20260709 13:12:21.017432   817 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.000s
I20260709 13:12:21.017995   817 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:21.021011   705 consensus_meta.cc:287] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.004s
I20260709 13:12:21.021500   705 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.952283   704 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:12:20.920814   852 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.024549   669 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.025549 29002 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:21.000707 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:20.957423   811 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:20.999356 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:20.923089 32630 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:12:20.932652   683 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:20.993883 29078 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:20.993959   810 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:21.030107 29711 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 6: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:12:21.037334 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 98598.9us
W20260709 13:12:21.038015 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.005949   853 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:21.005841   819 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:20.967235   783 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:12:21.025488   851 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.037096   765 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:12:21.002532   537 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.041031 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:21.062410 29003 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:21.029633   811 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:20.983780   814 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.058881   851 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.059949   679 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.046670   683 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:21.065402   530 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.061717 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:21.059048   782 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.163265   782 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.059433   819 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.073268 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.164793   819 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:21.165951 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.166131   576 raft_consensus.cc:3055] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:21.166488   576 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:21.167047 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:21.167745 29085 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:21.167327   576 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.073552   813 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:21.068996   853 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:21.168637 29709 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:21.168727   813 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.171619   813 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.172124 29156 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:21.081924 29151 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:21.089661   811 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:21.115108 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:21.077126 29715 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:21.093561 32630 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:21.094720 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:21.096621 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:21.116076 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.101922 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:21.042474 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.045174 28906 catalog_manager.cc:5697] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 5 to 6, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 6 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:21.102735   817 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.103386   828 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.188633 29713 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:21.105494 29154 raft_consensus.cc:1240] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:21.105716 29152 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:21.107281 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 150507us
W20260709 13:12:21.188907 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 147961us
W20260709 13:12:21.189929 29048 leader_election.cc:336] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.704s (SENT)
I20260709 13:12:21.190327 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:21.179843 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.109165 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:21.114950 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:21.072232 29157 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:12:21.119561 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:12:21.122025 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.159219 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:21.102358 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:21.192775 29712 raft_consensus.cc:2393] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:21.193353 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 96806.9us
I20260709 13:12:21.184671 29158 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:21.114317   851 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:12:21.117713   658 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:12:21.113481   865 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.196204   520 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:21.196105   865 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.109841   705 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.110584   679 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.198292   865 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:21.065620   537 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.199497   679 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:21.068670 29076 raft_consensus.cc:2410] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:21.078869   783 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:21.200708   783 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:21.201514   783 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.202473   783 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:21.102440 29051 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:21.204914 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 102591us
I20260709 13:12:21.190405 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:21.205435 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.103760   780 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:21.205894 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.206517 29051 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:21.208487 29711 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:21.208726 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:21.212423   836 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:21.215637   780 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.216461   875 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:21.212802 32630 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.217942   780 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.151648   810 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.219013 32630 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.143566   814 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.220245   679 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:21.220811   810 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.223203 29151 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:12:21.223686 29151 pending_rounds.cc:85] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 6
I20260709 13:12:21.223963 29151 pending_rounds.cc:107] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:12:21.224262   537 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:21.226604   828 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.214648 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:21.224283 29151 raft_consensus.cc:2889] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:21.236341 28906 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 4 to 7, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 7 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:21.239274 28906 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 5 to 6, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 6 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:21.264859 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:21.269511 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:21.270823 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:21.274868 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:21.275597 29004 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:21.292171 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.293092 29081 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:21.293845 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.295197 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.297066 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.298208 29081 raft_consensus.cc:2393] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:21.299431 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:21.305258 29081 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:12:21.323385   829 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.324818 29010 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:21.326030   829 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.328226 29081 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:21.337141 29150 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:21.439431 29008 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:21.469429 29158 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:21.494563 29001 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:12:21.506675   819 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.509127   819 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.541059   853 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.552409   853 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.608371   755 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:21.608943   755 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:21.610425   755 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.612764   755 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:21.622308 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 415815us
I20260709 13:12:21.625239 29006 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:21.637172 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:21.638398 29005 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:21.652485 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 462528us
W20260709 13:12:21.653395 29048 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.762s (SENT)
I20260709 13:12:21.660704 29079 raft_consensus.cc:2368] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11.
I20260709 13:12:21.666497 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.676944   703 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:21.677680   703 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.661443 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:21.694864 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 480228us
I20260709 13:12:21.699177 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:12:21.699647 28975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.016s	sys 0.012s
I20260709 13:12:21.702600 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:21.706971 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:12:21.699967 28975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.016s	sys 0.012s
W20260709 13:12:21.712996 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 738317us
I20260709 13:12:21.714063 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:21.718768   780 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:21.719549   780 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:21.720099 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:21.721156 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 97574us
I20260709 13:12:21.721935 29051 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:21.723147 29051 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:21.727432   819 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:21.728742   819 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:21.729521   819 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:21.730059   819 raft_consensus.cc:697] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:21.732226   777 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:21.732714   777 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:21.733639   777 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.735069   777 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:21.737819   837 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.738056 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:21.738343   837 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.738835 29712 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:21.740406 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.740774   836 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:21.741647   836 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.743358   692 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:21.743757   692 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:21.750622   692 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.731155   819 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.751991   692 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:21.744741   837 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.746369   836 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:21.751116   780 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.745508 29085 pending_rounds.cc:85] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 4
I20260709 13:12:21.762576 29085 pending_rounds.cc:107] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:12:21.762890 29085 raft_consensus.cc:2889] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:21.763962   837 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:21.759366 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.765173 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.766079 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 112736us
I20260709 13:12:21.714701   703 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.767033 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:21.766782   780 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.776564 29682 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:21.783319 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:21.791230 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:21.793709 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:21.811139   836 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:21.828400   836 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:21.828995   836 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:21.830029   836 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:21.832072 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:21.837177   523 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.837756   523 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.838838 32570 raft_consensus.cc:3055] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:21.839318 32570 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:21.840773 32570 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:21.842253 32570 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.843276 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.849181 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:21.849928 29078 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:21.851186 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:21.852120 29078 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
W20260709 13:12:21.858340 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 81853.7us
I20260709 13:12:21.863858   844 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:21.876442   780 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.877430   780 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:21.879395   523 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.887869 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.824712   755 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:21.877820   780 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:21.891484 28896 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.435s	user 0.001s	sys 0.002s
I20260709 13:12:21.907127 29078 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:21.917192 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63217.6us
I20260709 13:12:21.917855 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2023) took 1140 ms. Trace:
I20260709 13:12:21.887184   523 consensus_queue.cc:1059] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:12:21.920516 29077 rpcz_store.cc:276] 0709 13:12:20.777323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.779146 (+  1823us) service_pool.cc:224] Handling call
0709 13:12:20.779578 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:20.779810 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:20.780286 (+   476us) log.cc:844] Serialized 14 byte log entry
0709 13:12:21.664859 (+884573us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b18003e6bc0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f41192393c7 00007f411584b9a7 00007f411585b0bf 00007f411585b038
0709 13:12:21.793331 (+128472us) log.cc:844] Serialized 14 byte log entry
0709 13:12:21.917519 (+124188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:21.917575 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:21.917618 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.917688 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:21.917799 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33501,"prepare.queue_time_us":991,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":83794,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":6991,"spinlock_wait_cycles":5466368,"thread_start_us":374851,"threads_started":3,"wal-append.queue_time_us":237017}
I20260709 13:12:21.927925 29078 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:22.002328   782 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.017484 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:21.980911   877 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:12:22.018697 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:21.964169   881 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:21.970799   819 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.918493   755 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.071556   881 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.073625   755 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.918184   837 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:22.019747 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:22.075068 29078 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:22.075664 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1582) took 1078 ms. Trace:
I20260709 13:12:22.076239 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:22.077387 29081 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:22.078147 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 219064us
I20260709 13:12:22.077944 29078 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:22.071468   819 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.080617   881 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.082701   819 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:21.145684   849 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.083503   704 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:22.083878   704 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:22.084080   755 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:21.914248   780 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:22.084704   849 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:21.044364 28902 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52676 (request call id 244) took 22846 ms (client timeout 15000 ms). Trace:
W20260709 13:12:22.085142 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 291509us
I20260709 13:12:22.027655   465 raft_consensus.cc:3055] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:12:22.069574   822 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.472s	user 0.024s	sys 0.000s
I20260709 13:12:22.077977 29079 rpcz_store.cc:276] 0709 13:12:20.997122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.179568 (+182446us) service_pool.cc:224] Handling call
0709 13:12:22.075457 (+895889us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400199954. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:22.075601 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078674304}
W20260709 13:12:22.024304   811 consensus_meta.cc:287] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.826s	user 0.006s	sys 0.006s
I20260709 13:12:22.057407   669 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:22.085450 29007 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:22.086472 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.087230 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.090032 29078 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:12:22.097046 29078 pending_rounds.cc:85] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 5
I20260709 13:12:22.097273 29078 pending_rounds.cc:107] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:12:22.097481 29078 raft_consensus.cc:2889] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:22.084720   704 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.097456 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:22.090654 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2038) took 1148 ms. Trace:
I20260709 13:12:22.099700 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:22.101436 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.101539 29079 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:12:22.099679 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.102283   811 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.106043   811 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.106201   826 consensus_queue.cc:1059] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:22.103739   887 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:12:22.107194   826 raft_consensus.cc:3055] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:22.107566   826 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:22.107760   576 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.089493   465 raft_consensus.cc:740] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:22.111501   465 consensus_queue.cc:260] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.112504   465 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.096223   669 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:22.116955 32630 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:22.117097   669 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.118157   669 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.117548 32630 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:22.089586   892 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:22.122032   892 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:22.122470 32630 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.122828   892 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.102705 29079 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:22.098955 29084 rpcz_store.cc:276] 0709 13:12:20.942033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.976637 (+ 34604us) service_pool.cc:224] Handling call
0709 13:12:20.976931 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:20.977116 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:21.793322 (+816206us) log.cc:844] Serialized 14 byte log entry
0709 13:12:22.090405 (+297083us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:22.090455 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:22.090499 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.090540 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:22.090615 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":692574,"raft.queue_time_us":4181,"thread_start_us":359611,"threads_started":2,"wal-append.queue_time_us":252551}
I20260709 13:12:22.121155 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.127434 29009 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:22.134016 29083 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:22.123720   704 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.129240 29079 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.150164 29048 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.138919   891 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:22.129300   819 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:22.153828   826 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.155104   826 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.154311   881 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.124766   892 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.138978   836 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.124035   894 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:22.156308 29079 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:22.157042 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.129402   822 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:22.163930 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 449937us
I20260709 13:12:22.160490   894 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:22.166293   852 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.098s	user 0.013s	sys 0.007s
I20260709 13:12:22.166491   894 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:22.166649   852 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.099s	user 0.013s	sys 0.007s
I20260709 13:12:22.167166   875 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:22.167574   875 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:22.165782 29714 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:22.168612 29009 raft_consensus.cc:1240] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:22.169906 29005 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:22.170054 29714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 523) took 1061 ms. Trace:
I20260709 13:12:22.170446 29714 rpcz_store.cc:276] 0709 13:12:21.108786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.108909 (+   123us) service_pool.cc:224] Handling call
0709 13:12:22.169899 (+1060990us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b540021f554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:22.170002 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2225085696}
I20260709 13:12:22.172602   853 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.173065   853 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.173224   780 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.175017   853 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.163908   836 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.176501   537 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:22.177397 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:22.177901 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.177459   537 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 8.11, Last appended by leader: 11, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:22.178233 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:22.178797 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.179472   836 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.170725   828 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:22.180246   828 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.182246   828 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:22.183182 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.183277 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2279) took 1134 ms. Trace:
W20260709 13:12:22.184032 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:22.175818 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 985369us
I20260709 13:12:22.184262 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:22.185319 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:22.185389 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:22.186935 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:22.187805 29008 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:22.183655 29007 rpcz_store.cc:276] 0709 13:12:21.049172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.049350 (+   178us) service_pool.cc:224] Handling call
0709 13:12:21.049784 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:22.096327 (+1046543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.183084 (+ 86757us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001860d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:22.183226 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2381311872}
I20260709 13:12:22.202715 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.158375   829 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:22.205869 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:22.085376 28902 rpcz_store.cc:269] 0709 13:11:58.196765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.197075 (+   310us) service_pool.cc:224] Handling call
0709 13:12:21.032529 (+22835454us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:12.430024 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:12.432468 (+  2444us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:12.432525 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:12:12.432546 (+    21us) tablet.cc:662] Decoding operations
0709 13:12:12.461349 (+ 28803us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:12.461452 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:12.461484 (+    32us) tablet.cc:685] Acquiring locks for 26 operations
0709 13:12:12.568577 (+107093us) tablet.cc:701] Row locks acquired
0709 13:12:12.605409 (+ 36832us) write_op.cc:260] PREPARE: finished
0709 13:12:12.637296 (+ 31887us) write_op.cc:270] Start()
0709 13:12:12.798844 (+161548us) write_op.cc:276] Timestamp: P: 1783602732620936 usec, L: 0
0709 13:12:12.810648 (+ 11804us) op_driver.cc:348] REPLICATION: starting
0709 13:12:12.948935 (+138287us) log.cc:844] Serialized 10243 byte log entry
0709 13:12:13.554340 (+605405us) op_driver.cc:464] REPLICATION: finished
0709 13:12:13.603424 (+ 49084us) write_op.cc:301] APPLY: starting
0709 13:12:13.603534 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:13.604160 (+   626us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:13.604325 (+   165us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:13.610174 (+  5849us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:13.610786 (+   612us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=26
0709 13:12:13.610859 (+    73us) write_op.cc:312] APPLY: finished
0709 13:12:13.611615 (+   756us) log.cc:844] Serialized 176 byte log entry
0709 13:12:13.611822 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:13.615426 (+  3604us) write_op.cc:454] Released schema lock
0709 13:12:13.615548 (+   122us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:13.616127 (+   579us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:13.616168 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":129357,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":48794,"apply.run_cpu_time_us":7356,"apply.run_wall_time_us":12638,"num_ops":26,"prepare.queue_time_us":81361,"prepare.run_cpu_time_us":70212,"prepare.run_wall_time_us":828823,"raft.queue_time_us":128244,"raft.run_cpu_time_us":12418,"raft.run_wall_time_us":49726,"replication_time_us":743484,"spinlock_wait_cycles":5888,"thread_start_us":532627,"threads_started":4,"wal-append.queue_time_us":400954,"wal-append.run_cpu_time_us":22839,"wal-append.run_wall_time_us":1272873}]]}
I20260709 13:12:22.214488 28975 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:22.167930   875 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.175959   780 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.165675   897 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.221033 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.203649   853 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.221033 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.205343   849 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:22.213419   679 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:22.208822   520 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.232141 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.228715 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.205205   658 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:22.207801   865 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:22.231246 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.229566 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.235368 29005 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:22.235909 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:22.241875 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.251176 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.251273 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.251914 29008 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:22.252786 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.221459   692 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:22.252867 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.254736 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.260368 29008 raft_consensus.cc:2393] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:22.226567   817 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:22.217557   816 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:12:22.261667 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.262367 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.249361   829 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:22.274197 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.264125 29005 raft_consensus.cc:2393] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:22.275646 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.264554 29010 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:12:22.265163 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.276489 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.215415   813 consensus_queue.cc:1059] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:12:22.277390 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 91115.9us
I20260709 13:12:22.269495 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.218858   811 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 13:12:22.256814 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08000cc020 after lost signal to thread 27904
I20260709 13:12:22.277411 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.268373 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:22.302579 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08000569c0 after lost signal to thread 28942
I20260709 13:12:22.253017   679 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.278184 29005 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:22.233459   853 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.262283 32570 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.277527 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:22.304009 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 118742us
W20260709 13:12:22.304752 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1b20 after lost signal to thread 29034
I20260709 13:12:22.305641 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.248953   865 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.308995 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.226923   897 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:22.310586 29031 debug-util.cc:398] Leaking SignalData structure 0x7b0800401c40 after lost signal to thread 29035
I20260709 13:12:22.316341 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.317057 29010 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:22.317571 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.309492 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.250104   658 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.319115 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.321887 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.322280 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.322861 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.328819 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.275699   817 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:22.324071 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.341229 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.342342 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.339919 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:22.343520 29084 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:22.345360 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.347570 29084 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:22.300813   829 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:22.349188 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.349623 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:22.350479 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.355742 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.355996 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.356374 29010 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:22.309609   780 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.341171 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:22.360795 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.354372 29078 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:22.350512 29710 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:22.337258   836 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.358566 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.378050 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.378316 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.355800 29005 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:22.379830 29005 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:22.359370 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.360031 29007 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:22.380481 29009 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:22.363095 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.321113   816 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.364651 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.364379 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.368779 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.308862   881 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.371124 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.237032 28896 maintenance_manager.cc:643] P c44d1d5117734537b7202f74e0c1e920: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 5.560s	user 0.118s	sys 0.081s Metrics: {"bytes_written":29267,"cfile_init":1,"dirs.queue_time_us":466109,"dirs.run_cpu_time_us":453,"dirs.run_wall_time_us":1420,"drs_written":1,"lbm_read_time_us":411,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4345,"lbm_writes_lt_1ms":29,"peak_mem_usage":0,"raft.queue_time_us":11295,"raft.run_cpu_time_us":2408,"raft.run_wall_time_us":9803,"rows_written":63,"thread_start_us":32651,"threads_started":3,"wal-append.queue_time_us":433881}
I20260709 13:12:22.371385 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.399959 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.321018 32570 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.316685   679 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.355302   852 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:22.384454 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:22.364338   819 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:22.359162   813 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:12:22.375388 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2036) took 1454 ms. Trace:
I20260709 13:12:22.367199   891 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:22.385419   903 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.378911   894 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.413575   836 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.417608 29082 rpcz_store.cc:276] 0709 13:12:20.929143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.985587 (+ 56444us) service_pool.cc:224] Handling call
0709 13:12:20.994179 (+  8592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:20.994446 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:22.206653 (+1212207us) log.cc:844] Serialized 14 byte log entry
0709 13:12:22.375010 (+168357us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:22.375074 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:22.375131 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.375208 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:22.375328 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1118580,"raft.queue_time_us":4306,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":13478,"thread_start_us":241954,"threads_started":2,"wal-append.queue_time_us":199852}
I20260709 13:12:22.414975   819 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.467541 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.401345   816 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.482271 29155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:22.482838 29155 raft_consensus.cc:606] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.483170 29155 raft_consensus.cc:614] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:22.408808   881 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.486375 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.486981 29154 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:22.417062   814 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:12:22.419317 32630 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:12:22.469426   903 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.475540   829 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.506129   903 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:22.506287   765 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:22.506672   765 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:22.507867   894 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.480417   838 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:22.509086 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 595) took 1849 ms. Trace:
I20260709 13:12:22.509200 32570 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.509518 29152 rpcz_store.cc:276] 0709 13:12:20.659819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.679022 (+ 19203us) service_pool.cc:224] Handling call
0709 13:12:20.679319 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.189244 (+509925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.508857 (+1319613us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54003ecbd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:22.509007 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980063616}
I20260709 13:12:22.510155 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.510820 29155 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.512425 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:22.513108 29077 raft_consensus.cc:2393] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:22.514822   907 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:22.515352   907 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.491137   679 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:22.515859   874 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:22.476399   817 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.516412   874 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.517409   907 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.495570   658 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.496337   814 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:22.508806   838 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.518512 29710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 820) took 1595 ms. Trace:
I20260709 13:12:22.489893 28897 maintenance_manager.cc:419] P c44d1d5117734537b7202f74e0c1e920: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8132 bytes on disk
I20260709 13:12:22.506672   903 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:22.541954 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.524703 29155 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:22.481235   819 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.526381 29710 rpcz_store.cc:276] 0709 13:12:20.923577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.923684 (+   107us) service_pool.cc:224] Handling call
0709 13:12:20.943174 (+ 19490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:22.370257 (+1427083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.518260 (+148003us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400577bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:22.518419 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3236045184}
W20260709 13:12:22.527031 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 156162us
I20260709 13:12:22.559890 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.507416   765 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.528882 29004 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:22.532814 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.518443   874 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:22.521739 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 157477us
I20260709 13:12:22.564625 29159 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:22.540465 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.519210   777 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:22.551458 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.565544 29151 raft_consensus.cc:2393] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:22.542622 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.566617 29151 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:22.565814 29156 raft_consensus.cc:2368] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12.
I20260709 13:12:22.567804 29151 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:12:22.545687 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:22.570685 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:22.562353 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 199339us
I20260709 13:12:22.563323 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.577184 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.578531 29159 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:22.579638   912 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:22.579749 28896 maintenance_manager.cc:643] P c44d1d5117734537b7202f74e0c1e920: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.014s	user 0.001s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":345,"lbm_reads_lt_1ms":4}
I20260709 13:12:22.580302 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.580513   537 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:22.581557   826 consensus_queue.cc:1059] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:12:22.582540 29083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 2022) took 1810 ms (client timeout 1949 ms). Trace:
W20260709 13:12:22.583006 29083 rpcz_store.cc:269] 0709 13:12:20.776991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.777143 (+   152us) service_pool.cc:224] Handling call
0709 13:12:22.582377 (+1805234us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54000c8ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:22.582467 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3088258816}
I20260709 13:12:22.528342   837 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:22.572450 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.574420 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.566798 29154 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:22.575340 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.571481 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.582628   826 raft_consensus.cc:3055] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:22.528698   814 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:22.576668 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:22.587881 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:22.588627 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.537629   891 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:22.542266   903 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:22.601977 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.580257   912 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:22.619937 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45623.8us
I20260709 13:12:22.616816   765 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.631583 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.632210 29007 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:22.632544 29003 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:22.632463 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.632927 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.633525 29007 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:22.583025   683 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:12:22.575636   822 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:22.636808 29051 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:22.633378 29009 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:22.589092   658 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:12:22.638680 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 50896.1us
I20260709 13:12:22.616525   865 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:22.658571 28975 leader_election.cc:400] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12.
I20260709 13:12:22.658999 28975 leader_election.cc:403] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:22.657529 29010 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:12:22.590409   853 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.643002 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.617973   810 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:22.643268 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1593) took 1607 ms. Trace:
I20260709 13:12:22.667482 29081 rpcz_store.cc:276] 0709 13:12:21.036145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.329097 (+292952us) service_pool.cc:224] Handling call
0709 13:12:22.077203 (+748106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:22.077771 (+   568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.635459 (+557688us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b5400199950. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:22.643215 (+  7756us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716590848}
I20260709 13:12:22.645924   827 consensus_queue.cc:1059] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:22.646200   576 consensus_queue.cc:1059] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:22.669166   838 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.668797   814 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.669817 29159 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:22.670383 29159 raft_consensus.cc:606] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.670894 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.671517 29153 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:22.635054   828 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:12:22.670116   814 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:22.672497   906 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:12:22.672345   658 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:12:22.673472   908 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:12:22.639482   822 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.674052   822 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:22.642041   537 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:22.676528   537 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.678082   903 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.678687   537 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.679097   903 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:22.651181 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 63607.1us
W20260709 13:12:22.680981 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.220s (SENT)
I20260709 13:12:22.681362 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.623412   904 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.616314   826 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:22.623553   891 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.619635   679 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:22.583751   777 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.669317   576 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:22.649958 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.626878   811 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:22.684639 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.685583 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:22.686467 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.593647   837 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.625665   907 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
W20260709 13:12:22.703436 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.686842   817 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:22.666177   909 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:22.706301 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:22.640367 32570 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:12:22.691793   913 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:22.682929   826 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:22.713006 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.720808 29155 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:22.712874 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 62988.4us
I20260709 13:12:22.722508 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.692425   853 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:12:22.717052 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 77221.4us
I20260709 13:12:22.698653   576 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:22.719385 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:22.711966 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 135335us
I20260709 13:12:22.726241 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.720075   874 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:22.727105 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:22.714241   530 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.718171   828 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:22.718305   913 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.746937   913 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:22.748456 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:22.716414   755 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:22.740857 29122 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:22.751353 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:22.741660   828 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.699764   679 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.757119 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:22.759681 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.761132 28976 leader_election.cc:400] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11.
I20260709 13:12:22.762048 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.741199 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:22.725924   892 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:22.724712   816 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.739696 29155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:22.772114 29155 raft_consensus.cc:606] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.772468 29155 raft_consensus.cc:614] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:22.740182   692 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:22.736531   919 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:22.744496   924 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:22.734115   658 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.768463 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:22.718169   819 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:22.749833 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:22.777362 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 50335.8us
W20260709 13:12:22.777787 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.777817 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:22.778833 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:22.725641   827 consensus_queue.cc:1059] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:12:22.755244   881 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.758778   828 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:22.774881   865 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:22.719198   909 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.759821   891 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.792279 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:22.787978 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.776276   520 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:22.775547   907 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:22.758606   873 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:12:22.726418   912 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.781037   816 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.802197 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.804491 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:22.804880 29155 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:22.805137 29708 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:22.799480 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:22.811862   819 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.780447   576 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.801399   814 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:12:22.813441   576 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.794552   865 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.798792 32570 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:22.801043   919 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:22.792140   923 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:22.797226   520 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.798980   669 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:22.820812 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:22.830802 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:22.791620 32630 consensus_queue.cc:1059] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:12:22.841009   679 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.841112   898 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.829926 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.844516   813 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:22.848698   837 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.827579 32570 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.851727   919 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.837442   932 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:12:22.854164 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 217396us
I20260709 13:12:22.876919 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:22.799204   922 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:12:22.854873 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58833.2us
W20260709 13:12:22.859331 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 59918.1us
I20260709 13:12:22.860340 29153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:22.828011   865 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:22.840176 29717 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:22.849406   520 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:22.829583 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:22.885617 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 56145.3us
I20260709 13:12:22.872290   909 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:22.889969 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:22.890511 29713 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:22.873775   897 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.851889   923 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:22.849577   669 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:22.877794 29082 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:22.875234   813 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:22.880774 29153 raft_consensus.cc:606] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.880937 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:22.863547   777 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.882206 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:22.867738   875 consensus_meta.cc:287] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.641s	user 0.008s	sys 0.002s
I20260709 13:12:22.887356 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:22.887666 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08003455c0 after lost signal to thread 29109
W20260709 13:12:22.957945 29031 debug-util.cc:398] Leaking SignalData structure 0x7b080036f9a0 after lost signal to thread 29110
W20260709 13:12:22.878968 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.868520 32570 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.881507   922 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:22.896282 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.968251   922 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:22.896850 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:22.968962 29717 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:22.865634   898 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.874819   919 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.902442 29716 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:22.902779   837 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:22.972340   837 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.972832   837 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:22.903424   782 raft_consensus.cc:3055] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:22.888374 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:22.908514   923 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:22.974206   782 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:22.974431 29712 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:22.975140   782 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.976490   782 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.977594   576 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.915797 29154 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:12:22.916460   780 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.000s
I20260709 13:12:22.978331   780 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.981359   576 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.982863   780 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:22.879400   932 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:22.988310 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.989174   780 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:22.989558   780 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.989872   780 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:22.997138   898 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:22.998765 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.999428 29154 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:23.000676 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:23.001619   898 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:23.002190   898 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.002640   898 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.994983   923 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.008222   465 raft_consensus.cc:3055] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:23.008668   465 raft_consensus.cc:740] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:23.009627   465 consensus_queue.cc:260] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.011494   923 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.012929   704 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:23.013362   704 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:23.014187   704 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.015488   704 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:23.016342   907 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:23.016825   907 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:23.017670   907 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.946164   865 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.956313   913 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:22.959134 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08003a24a0 after lost signal to thread 29183
I20260709 13:12:22.957981 29153 raft_consensus.cc:614] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:22.936225 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:22.939159 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:22.961555 29710 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:12:22.962739 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 81870.3us
I20260709 13:12:22.960211 29159 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:23.035858 29158 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
W20260709 13:12:22.888031 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 139471us
I20260709 13:12:22.962363   875 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.965967   679 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:22.914867   669 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.036561 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:23.037081 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 100933us
I20260709 13:12:23.038411 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:23.041729 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:23.019232   907 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:23.045868 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:22.920270   783 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:23.047209 29082 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
W20260709 13:12:23.038411 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08003539e0 after lost signal to thread 29741
W20260709 13:12:23.050585 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.050645 29031 debug-util.cc:398] Leaking SignalData structure 0x7b0800412a40 after lost signal to thread 29742
W20260709 13:12:23.051342 29031 debug-util.cc:398] Leaking SignalData structure 0x7b080037a6c0 after lost signal to thread 635
I20260709 13:12:23.051578   810 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:23.051805 29031 debug-util.cc:398] Leaking SignalData structure 0x7b080027d2a0 after lost signal to thread 754
W20260709 13:12:23.052316 29031 debug-util.cc:398] Leaking SignalData structure 0x7b080035f080 after lost signal to thread 793
I20260709 13:12:23.052210   810 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:23.052783 29031 debug-util.cc:398] Leaking SignalData structure 0x7b0800411820 after lost signal to thread 800
I20260709 13:12:23.054229   810 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.046293   833 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:23.037509   777 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.048466   827 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:23.001897   817 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12.
I20260709 13:12:23.064769 29001 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:23.066383   576 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.066675   931 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:23.064684   827 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.067095   865 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.068943 29159 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:23.069486 29159 raft_consensus.cc:606] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:23.069809 29159 raft_consensus.cc:614] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:23.070137   875 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.070601   898 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.067269   827 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:23.072172   932 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:23.056742   923 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:23.072981   923 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:23.073360   923 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:23.071276 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:23.062853   833 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.076465 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:23.077023   833 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:23.076501   777 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.078020   780 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.079661   777 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:23.080314 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.080904   669 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:23.081883   780 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:23.081616 29708 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.083308 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:23.087344 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.090826 29051 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:23.093538 29159 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:23.067152   931 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.093698 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:23.094044   931 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:23.094353 29714 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:23.095132   898 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.096474   933 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:23.097558   933 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:23.097870   933 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:23.098613   933 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.102485 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:23.103080 29001 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:22.919966   828 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:22.943406   822 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.104075 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:23.099864   933 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:23.115756 29079 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:23.049276   783 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:23.107137 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:23.113027 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 351029us
I20260709 13:12:23.130232 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.130451 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.124873 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.105886 29714 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:23.131806   679 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:23.133724 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.134243 29153 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:23.136593 29079 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:23.130803 29159 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:23.127548 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2026) took 2236 ms. Trace:
I20260709 13:12:23.139444 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:23.140895 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.140842 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.142448 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.145032 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.140275 29084 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:23.149786 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.150877 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.151219 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:23.151863 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:23.114001   777 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:23.112428   465 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:23.138918 29080 rpcz_store.cc:276] 0709 13:12:20.891221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.896129 (+  4908us) service_pool.cc:224] Handling call
0709 13:12:20.918021 (+ 21892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:20.918204 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:20.949987 (+ 31783us) log.cc:844] Serialized 14 byte log entry
0709 13:12:22.890659 (+1940672us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.126131 (+235472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:23.126192 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:23.127325 (+  1133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.127401 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.127500 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1823253,"raft.queue_time_us":44005,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":2186,"thread_start_us":303702,"threads_started":2,"wal-append.queue_time_us":254414,"wal-append.run_cpu_time_us":1205,"wal-append.run_wall_time_us":1001398}
I20260709 13:12:23.151885 29078 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:23.155115 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.155840 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 49930.1us
I20260709 13:12:23.159613 29680 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:23.155757 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.159852 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:23.160425 29084 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:23.161334 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:23.161021 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 280063us
I20260709 13:12:23.161738 29084 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:23.149253   874 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:23.166893 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:23.162999 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.168965 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.169950 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:23.171845 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:23.174068 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.177004 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:23.134438   913 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.133435   881 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.177280 29715 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:23.113116   894 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:12:23.183909 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:23.154510   875 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:23.198933   934 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:23.199491   927 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:23.246488   927 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.246994   927 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:23.163283 32630 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.136232   814 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:12:23.250252 32630 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.214767 29010 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:23.183805 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:23.214510   783 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.198938   911 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:23.217571   520 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.263095 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.219388   837 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.221351   919 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.141601 32570 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.176743   810 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:23.222239   931 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.219509   923 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.215207   922 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.224080   833 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.226301 29084 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:23.229383   822 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.186250 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:23.282704 29081 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:23.283073 29714 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:23.232960   828 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.235198 29078 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
W20260709 13:12:23.237005 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 68135.7us
W20260709 13:12:23.236974 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.292590 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.292671 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:23.238540 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.239127 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:23.294476 29005 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:23.240065 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.244570 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.103866   813 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:23.251550 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08003a9f60 after lost signal to thread 820
W20260709 13:12:23.296169 29031 debug-util.cc:398] Leaking SignalData structure 0x7b0800367300 after lost signal to thread 831
I20260709 13:12:23.296284 29003 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:23.296495 29031 debug-util.cc:398] Leaking SignalData structure 0x7b080023f780 after lost signal to thread 835
W20260709 13:12:23.296924 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08003ba580 after lost signal to thread 846
W20260709 13:12:23.297160 29031 debug-util.cc:398] Leaking SignalData structure 0x7b08002e4000 after lost signal to thread 848
W20260709 13:12:23.297479 29031 debug-util.cc:398] Leaking SignalData structure 0x7b0800350800 after lost signal to thread 856
I20260709 13:12:23.159027   777 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.300688 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.303125 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.304607 29084 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:12:23.308343 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.311450   907 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:23.280041   913 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:23.253134 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 79182.7us
I20260709 13:12:23.309885   704 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:23.256028 32630 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.321099   913 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.257716   894 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:23.321416 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.260213 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:23.260473 29156 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:23.187454   576 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:23.265272 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.266310 29153 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:23.334599 29152 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:23.267666 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:23.335227 29003 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 11. Preceding OpId from leader: term: 9 index: 12. (index mismatch)
I20260709 13:12:23.268420 29154 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:23.260372 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 76646.6us
I20260709 13:12:23.239800   827 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.262575   530 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:23.274377 29006 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:23.185102   881 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.274789   520 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.279661 29150 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:23.261229   911 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:23.262828   703 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.298326 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:23.299708 29713 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:23.312790 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:23.381264 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 83048.9us
I20260709 13:12:23.314688 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:23.318374   837 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:23.319931   922 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.257004   902 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:23.322038   894 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:23.254140 29159 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:23.327129 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.323391 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.336714   576 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.336987   934 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:23.341462   833 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.343166 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.343740 32630 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:23.344669 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.407661 29708 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:23.408424 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:23.346472 29078 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:23.347944   913 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.348428   938 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:23.409766 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:23.410425 29078 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:23.343449 29157 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:23.350418   931 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.303720   777 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:23.369524 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46754.8us
W20260709 13:12:23.413692 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 70588.7us
I20260709 13:12:23.414479 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.380352 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:23.381369 29010 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:23.309437   810 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:23.382197 29007 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:23.351562   907 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:23.386674 29158 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:23.398917 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.428402 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.428615 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.429049 29082 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:23.429307 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.429567 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.429809 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:23.430254 29079 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:23.401005 29712 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
W20260709 13:12:23.404767 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 77697.1us
I20260709 13:12:23.439988 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.417232 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:23.547982 29078 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:23.412531   822 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:23.424405 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.425954 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.549901 29008 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:23.403513   703 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.427395 29154 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:23.411971 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.398138   902 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:23.534936 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.535709 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.368323   904 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.402240   530 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.538478 29079 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:23.385743   932 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 13:12:23.540745 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 111538us
I20260709 13:12:23.583263 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:23.396826   838 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:23.407436   576 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:23.541345 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.586678 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 713) took 2380 ms. Trace:
I20260709 13:12:23.413514   892 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.542780 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:23.548992 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:23.426968 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.554126 29082 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
W20260709 13:12:23.536891 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 122490us
I20260709 13:12:23.425668   810 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.428977   922 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:23.407262   897 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.145s
I20260709 13:12:23.539103   828 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:23.425613   704 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 13:12:23.573256 29004 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:23.581691 29157 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:23.560451   683 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:23.583899 29153 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:23.551365   852 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.583622 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.581676 29010 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:23.550221   931 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.543880 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2041) took 1847 ms. Trace:
I20260709 13:12:23.553417   875 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:23.588591 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.606928 29151 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:23.562916 32570 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.587064 29076 rpcz_store.cc:276] 0709 13:12:21.206170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.510109 (+303939us) service_pool.cc:224] Handling call
0709 13:12:21.510388 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.510599 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:23.350471 (+1839872us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.586312 (+235841us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:23.586370 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:23.586418 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.586493 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.586624 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1839586,"prepare.queue_time_us":1867124,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":47,"thread_start_us":235470,"threads_started":1}
I20260709 13:12:23.596011 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.610423 29003 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:23.603315 29008 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:23.609864 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.621311 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.621371 29125 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.584481   780 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.632863 29125 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:23.565322   927 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.631439 29003 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:23.625682 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 82984.7us
I20260709 13:12:23.651826 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:23.605929 29085 rpcz_store.cc:276] 0709 13:12:21.696403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.714916 (+ 18513us) service_pool.cc:224] Handling call
0709 13:12:21.745316 (+ 30400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:21.767481 (+ 22165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:21.767516 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:21.768142 (+   626us) log.cc:844] Serialized 52 byte log entry
0709 13:12:21.814073 (+ 45931us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:23.351014 (+1536941us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.543501 (+192487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.543596 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.543651 (+    55us) raft_consensus.cc:1685] finished
0709 13:12:23.543669 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.543816 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1543145,"prepare.queue_time_us":1574617,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":8634,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":14047,"spinlock_wait_cycles":8135040,"thread_start_us":236104,"threads_started":2,"wal-append.queue_time_us":125401,"wal-append.run_cpu_time_us":7253,"wal-append.run_wall_time_us":1007698}
I20260709 13:12:23.590091   938 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.590590   836 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:23.593582   810 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:23.622473   912 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:23.625622   933 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:23.661342   814 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:23.668092 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.631982   849 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:23.587119   530 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.668084 29008 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:12:23.670531 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.697687 29153 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:23.698375   836 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.670308 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:23.701900 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:23.703035 29080 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:23.698885 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.589856   892 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:23.677014 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.605057   838 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:23.718403 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.760466 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.691809   938 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.810045   938 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:23.810582   933 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.811107   933 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.813128   933 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.813621   933 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.814059   933 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.664209   902 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.632704   931 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.763406   827 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.648198   875 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:23.764400 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:23.651142   837 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:23.693219   817 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
W20260709 13:12:23.765617 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60516.1us
I20260709 13:12:23.766347 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:23.698869   951 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:12:23.633973   922 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:23.650290   520 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:23.584242   894 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.823365 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.768181 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.647401   919 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.661048   935 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:23.787423 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 117174us
I20260709 13:12:23.799906 29004 raft_consensus.cc:2376] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11. Re-sending same reply.
I20260709 13:12:23.603875   904 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.854043 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:23.624647   852 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.819854 29010 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:23.821473 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.820353 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.861196 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:23.650686   780 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:23.767266   811 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:23.692451   903 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.140s
I20260709 13:12:23.836700 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.921657 29002 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:23.837371 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.825687   949 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 13:12:23.797878   912 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.819831   837 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.799577   537 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:23.816670   902 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:23.818320   833 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.864071 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.949955 29004 raft_consensus.cc:2410] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:23.816022   933 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.888919 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:23.953133 29003 raft_consensus.cc:2368] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:23.821441   931 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.956561   931 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:23.956872   927 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:23.957048   931 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:23.958105   952 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:12:23.958886   703 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.959215   931 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:23.961534   904 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:23.819450   875 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.965767 29077 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:23.968833 29083 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:12:23.918572 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54973.9us
I20260709 13:12:23.930588 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:23.931558 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 70493.3us
I20260709 13:12:23.976637 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.979262 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:23.933198 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:23.980418 29010 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:23.967895 29085 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:12:23.802287   955 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:23.938521 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:24.032575 29159 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:23.939966 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.033201 29153 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:23.835947   777 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.863873   817 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:23.947144 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:23.920718   919 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:23.975704 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:23.936173   828 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:23.938862   537 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:24.043761   828 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:23.984956 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:24.049145 29713 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:23.985889   822 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:24.051592 29084 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:23.985833   849 consensus_queue.cc:1059] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:23.987035 29004 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:23.921562   576 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.992486 29008 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:23.993278 32630 raft_consensus.cc:993] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:12:23.968638   833 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:23.993830 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:24.056741 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.995466   852 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.998453   530 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:23.934420   811 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.969370   838 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:23.930202   935 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:23.949967 32570 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:24.021179   959 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:24.022156 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:24.028622 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:24.042805 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 67174.2us
I20260709 13:12:23.939606   912 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:24.050012 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 689274us
I20260709 13:12:24.052117   537 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:24.027698   810 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.046976   933 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:24.050976   828 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:24.058367   765 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:24.068348 29076 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:24.071771 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:24.064451 32630 raft_consensus.cc:1081] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Signalling peer befda4a6c8344c26b0f139e4e1686b3b to start an election
W20260709 13:12:24.077682 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 375848us
I20260709 13:12:24.084640 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:24.087383 29085 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.087908 29002 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:24.042124   955 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.069470   530 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:24.090173 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:24.090512 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:24.090854 29714 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
I20260709 13:12:24.091270 29082 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:24.093109 29713 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:24.088624 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.066900   780 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.095263 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.095849 29079 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 6.
W20260709 13:12:24.078619   813 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.026s	sys 0.057s
I20260709 13:12:24.099368   576 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.101074 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:24.127301 29076 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:12:24.101531   530 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:24.097922 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:24.086027   903 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:24.106822 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.134709 29079 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:24.127720   813 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.026s	sys 0.058s
I20260709 13:12:24.111747 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:24.142196 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.113154   912 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.113665 29007 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:24.113955 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.114660   810 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.120749   926 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:24.121006   948 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.070255   959 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:24.070089 32570 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:24.125588 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.151794 29154 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:24.127988 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:24.106158 29085 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
W20260709 13:12:24.153159 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 206090us
I20260709 13:12:24.153192 29153 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:24.130141 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.087706   932 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:12:24.110256 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.134374 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.167025 29077 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:24.140450 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:24.088521   933 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:24.160632 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.138614   938 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:24.166970 29083 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:24.126878   822 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:24.175205   780 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:24.175884   965 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:24.175727   780 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:24.177158   780 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.132565 29048 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.144632   937 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:24.181011   937 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.183666   937 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.172374 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.184948 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.185539   937 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:24.186452   822 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:24.186865 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2061) took 1038 ms. Trace:
I20260709 13:12:24.187367 29080 rpcz_store.cc:276] 0709 13:12:23.148099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.223003 (+ 74904us) service_pool.cc:224] Handling call
0709 13:12:23.223338 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:23.790917 (+567579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.186633 (+395716us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b540034f154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:24.186798 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102750464}
I20260709 13:12:24.188354   937 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:24.189317   937 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.192261   960 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:24.193220   955 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.193650   836 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:24.193197 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60741.6us
I20260709 13:12:24.194674 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.194589   530 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.195240   911 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.195914 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.187850 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.196537   777 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.198534   811 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.199676 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:24.199916   833 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.200539 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:24.200697 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:24.201264 29714 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:24.202461 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:24.203387 29717 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:24.203186   933 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.146169   958 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:12:24.201287 29716 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:24.186916   822 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:24.236475 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:24.235886 28892 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 06ae4197fd784525b695a9b16c8094ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:24.242612 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.243307 29085 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 6.
I20260709 13:12:24.208206 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.148396   814 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:12:24.245834 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.251317 29078 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:24.191812   874 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.245258 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.247740 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.149780   909 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:12:24.157130   959 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:24.174338   938 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.172739   783 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:24.168994   935 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:24.221091 29683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.157007   813 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:12:24.296386 29082 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:24.254688 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.297204 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:24.255111 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.306480 29085 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:24.238989   958 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:24.238857   777 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:24.299893 29683 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:24.213860   955 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:24.246719   960 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:24.241353   836 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:24.308514 29683 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:24.309290 29078 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:24.308696 29077 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:24.317085 29682 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:24.319624 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:24.314653 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 67010us
I20260709 13:12:24.166442   948 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.299862   938 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:24.307585   897 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:24.308176   958 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:24.331918 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.316221   927 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:24.319583   926 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:24.318558   814 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:24.300309   874 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.316607   777 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:24.342150 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 104433us
I20260709 13:12:24.320858   903 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:24.310837   537 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.255823 32570 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.328878   576 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:24.328977   852 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:24.360296 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.331453   897 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.340725   814 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:24.358805 29048 leader_election.cc:400] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:24.391392 29048 leader_election.cc:403] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:12:24.392431 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 48867.8us
I20260709 13:12:24.334990   958 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.394608   919 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:12:24.363525 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 208245us
I20260709 13:12:24.320499   836 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.341104   926 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.396695   926 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:24.396780 29712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49414 (request call id 613) took 1246 ms (client timeout 1507 ms). Trace:
W20260709 13:12:24.397243 29712 rpcz_store.cc:269] 0709 13:12:23.150628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.151562 (+   934us) service_pool.cc:224] Handling call
0709 13:12:24.396615 (+1245053us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400502154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:24.396727 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570866944}
I20260709 13:12:24.321365   822 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.371881   537 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.372740 29155 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:24.357587   894 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:24.347414 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:24.379575 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.380440   967 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:24.381830   897 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:24.384491 29077 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:24.386991 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.390074   927 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:24.343763 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 199494us
I20260709 13:12:24.422005 29125 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.394750 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.364787 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.372706   903 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.410615 29154 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:24.392881   937 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:24.413095   836 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.413718   892 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.338313   875 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.338084   811 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.419108 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:24.366120   852 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.425282 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:24.441855 29009 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:24.426164 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:24.426651   919 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:24.411612 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:24.428862 29002 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:12:24.449182 29002 pending_rounds.cc:85] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 6
I20260709 13:12:24.449416 29002 pending_rounds.cc:107] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:12:24.449627 29002 raft_consensus.cc:2889] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:24.429395 29158 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:24.431598 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.450994 29008 raft_consensus.cc:2393] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:24.452745 29076 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:24.455111 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.457450 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.431686 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.432186 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.432264 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.458740 29010 raft_consensus.cc:2393] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:24.458842 29004 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:24.459187   972 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:24.459457 28975 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:24.459707   972 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.436856 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.460793 29154 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:24.433063 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.390731   777 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.464215 29003 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:24.464320   969 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.464848   969 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.466483   964 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:24.467077   969 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:24.458490 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.469880 29077 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:24.469694 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:24.473814 29158 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:24.437091   683 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:24.470881 29125 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.477725 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.455588   852 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.466248   790 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Committing config change with OpId 7.7: config changed from index 3 to 7, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:24.480616 29158 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:24.477350   683 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.429231   927 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.481076   971 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:24.482249   977 log.cc:927] Time spent T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:12:24.478510 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.456825   849 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:24.482365   919 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:24.484696 29083 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 3 to 7, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:12:24.491961 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.475414   780 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.440266   828 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.479264   833 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:24.495901 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.503479 29158 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:24.455099   811 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.454913   875 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.499298   537 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:24.494454   683 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:24.509245   933 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:24.511674 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.511873 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:24.500967   967 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.455801   814 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.498009   849 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:24.497188   919 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:24.510200   537 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.516750   933 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.520982   811 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.504528   971 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.525053   970 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:24.527328   790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:51794 (request call id 0) took 1677 ms. Trace:
W20260709 13:12:24.528177   974 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:12:24.529273   875 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.530902   828 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.505245   780 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.532632 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 728) took 1657 ms. Trace:
I20260709 13:12:24.611545 29081 rpcz_store.cc:276] 0709 13:12:22.875356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.881969 (+  6613us) service_pool.cc:224] Handling call
0709 13:12:24.532376 (+1650407us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b540034d854. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:24.532557 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745943424}
I20260709 13:12:24.535481 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.613099 29150 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:24.504753   833 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:24.549401 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:24.539021   926 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.510051   894 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:12:24.518136   874 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.539242   849 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:24.560817 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.543761   919 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:24.567624   963 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:12:24.567950   973 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:24.558647 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 99271us
I20260709 13:12:24.552479   814 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:24.544989   537 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:24.543411   822 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.514101   969 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:12:24.610134 29001 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:24.666487 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.570957   933 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:24.572387   970 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.512826   948 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:24.621536   979 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:12:24.617496   833 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:24.623625   971 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.625037   927 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.684235 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.685412 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:24.626983 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.686548 29005 raft_consensus.cc:2393] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:24.687274 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.687907 29005 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 8: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:24.627250   938 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:24.628578   981 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:12:24.633669   932 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:24.640538 29150 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:24.653371 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.694000 29158 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:12:24.694654 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 145312us
I20260709 13:12:24.663414 29076 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:24.695580 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:24.615407   811 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:24.671517 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 159682us
I20260709 13:12:24.673264 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:24.703408 29001 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:24.648610   960 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:24.680529   959 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.681388   826 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:24.683149 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.688597 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.710469 29005 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:24.669698 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2320) took 1244 ms. Trace:
I20260709 13:12:24.706552 29152 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:24.710382 29155 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:24.712291 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:24.714529   881 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:24.715684   983 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:12:24.716037 29083 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:24.717474 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.673915   970 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.690878 28975 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:24.711629 29007 rpcz_store.cc:276] 0709 13:12:23.425323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.426244 (+   921us) service_pool.cc:224] Handling call
0709 13:12:23.540734 (+114490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:24.146926 (+606192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.669482 (+522556us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b5400291654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:24.669645 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317172608}
I20260709 13:12:24.719314 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.693042   932 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:24.717031   986 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.693342   874 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.701642   990 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.668812   822 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.727720 29005 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:24.730572 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:24.739907 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.740537 29005 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:24.732457 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:24.741358 29083 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:12:24.708565   926 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.704113   927 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:12:24.747865 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.740185 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:24.707782   960 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:24.726894   828 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:24.721432   881 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:24.735121 29001 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:24.764211 29680 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:24.764704 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:24.765399   926 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:24.766430   926 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:24.767158 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:24.765803   998 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:24.727064   894 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:24.760684   994 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.761540 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:24.706238   811 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.731484   964 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:12:24.733013   982 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.767391   926 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 8.11, Last appended by leader: 11, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:24.756633 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 61158.3us
I20260709 13:12:24.785880 29710 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:24.728719   971 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:12:24.756690   992 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:24.572707   790 rpcz_store.cc:276] 0709 13:12:22.849969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.850106 (+   137us) service_pool.cc:224] Handling call
0709 13:12:23.080363 (+230257us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b54007411a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157b8ec4 00007f411583c8d7 00007f411584803c 00007f41158476b3 00007f411583f9fa 00007f411a8074c9 00007f4113d83c54 00007f4113d83be2 00007f4113d83b11 00007f4113d83a8c
0709 13:12:23.097111 (+ 16748us) log.cc:844] Serialized 494 byte log entry
0709 13:12:24.175180 (+1078069us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b54007411a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157b6dbe 00007f41157c1930 00007f41157c18d8 00007f41157c1858 00007f41157c1813 00007f41157c0a9c 00007f411b6ac626 00007f411b6a6dd8 00007f41157fd7cb 00007f41158002c0
0709 13:12:24.194012 (+ 18832us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b4c0027d540. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f4113b0da20 00007f4113b1ea29 00007f4113b1f13f 00007f4113ae32ab 00007f4113b10f3c 00007f4113b113fe 00007f4113d9e50b 00007f411579d381 00007f411579b26b 00007f41157a1d53 00007f41157a1cd9
0709 13:12:24.527222 (+333210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4738,"raft.queue_time_us":5026,"raft.run_cpu_time_us":6958,"raft.run_wall_time_us":1013422,"spinlock_wait_cycles":176300928,"thread_start_us":84291,"threads_started":1,"wal-append.queue_time_us":147502}
I20260709 13:12:24.758069   875 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:24.758540   960 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.762557   973 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:24.779500   704 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:24.760819   969 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:12:24.800678 29051 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:24.746273   874 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.793543   912 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:24.744443   986 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.747292   990 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:24.818338 29084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 740) took 1436 ms (client timeout 1516 ms). Trace:
W20260709 13:12:24.818760 29084 rpcz_store.cc:269] 0709 13:12:23.382411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.408192 (+ 25781us) service_pool.cc:224] Handling call
0709 13:12:24.818138 (+1409946us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54000244d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:24.818277 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477219072}
I20260709 13:12:24.819552   992 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:24.819903   990 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:24.820492 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 13:12:24.815889   995 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.821434   995 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.823625   995 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:24.777614   881 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.789721   894 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.809921   704 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.809039   875 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.793669   932 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.778450   994 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.843648   990 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:24.844373   902 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:24.828562   912 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.942719 29084 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:12:24.849056   837 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:24.893596 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.897204   874 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.901605 29083 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:24.927462 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:24.929438 29005 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:24.901489   704 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.803193   982 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.955122   881 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.956071   537 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.957484   912 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.958197   790 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:24.963119   933 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.964708 29085 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:24.967300 29080 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:24.902113   985 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:12:24.939901 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 131488us
W20260709 13:12:24.977598 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.920678   828 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:24.832914   991 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:24.901240   875 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:24.976951 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.944078 29076 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:44428
I20260709 13:12:24.991813 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:24.942096   530 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:24.982298   811 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.942751   833 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:24.972586   938 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:24.978525   932 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:25.012964 29156 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:12:25.002487 29076 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:25.013732 29122 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:25.002978 29154 raft_consensus.cc:2393] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:24.996482 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 256395us
I20260709 13:12:25.004720 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:25.015452 29155 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 9.
I20260709 13:12:25.014004 29076 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:25.006743 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:25.002157 29152 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:25.023419 29151 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:25.007812   833 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:24.990295   959 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.005069   683 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.947345   902 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:24.963490   970 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.038887 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:25.042907 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:25.054625 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:25.055198 29154 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:25.056464 29154 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:25.057520 29154 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 13:12:25.029646   958 consensus_meta.cc:287] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.000s
I20260709 13:12:24.998102   985 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:25.003180   530 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:25.000501   828 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:25.065716 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:25.071117 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:25.071592 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.005940   994 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:24.938025   852 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.001833   537 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:25.061285 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 117900us
I20260709 13:12:25.000782   991 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:25.027925   822 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.041639   814 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:25.054942   996 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:25.088455 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:25.089104 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2308) took 1913 ms. Trace:
I20260709 13:12:25.103577 29006 rpcz_store.cc:276] 0709 13:12:23.175514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.175635 (+   121us) service_pool.cc:224] Handling call
0709 13:12:23.175859 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:23.368852 (+192993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.086440 (+1717588us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b540032cdd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:25.089045 (+  2605us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226176384}
I20260709 13:12:24.979651   973 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:25.077695   897 consensus_meta.cc:287] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.000s
I20260709 13:12:25.104391 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:25.064811  1002 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.110242 29076 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.116817   814 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:25.116946   897 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.117867   814 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.118985   897 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.126394 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.129732 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.131170 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:25.132169 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 329606us
I20260709 13:12:25.138620 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.139523 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:25.140403 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.141052 29081 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:25.141940 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.142475 29081 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:25.143142 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.144894 29084 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:25.087311   894 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.147799 29683 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:25.150094 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:25.170077 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.120306  1002 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.177275 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.112238   986 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.173041 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.185294 29080 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:25.115947   926 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:25.164637   994 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.207954 29084 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:25.227648 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:25.228375 29084 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:25.208462 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:25.209646 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.092381   995 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 13:12:25.173008   897 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:25.215416  1001 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:25.176594   523 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.147883   996 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:25.067889   958 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.176879   987 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:25.266688 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:25.267484 29083 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 7: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:25.207149   986 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.273036  1006 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:25.174055 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 250877us
I20260709 13:12:25.175422   874 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:25.234663 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 44452.4us
I20260709 13:12:25.188220   964 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:25.263872 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 463253us
I20260709 13:12:25.279037 29051 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:25.243692  1001 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.279878 29051 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:25.147157   530 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.235627   897 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.174269   985 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.286890   881 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:25.273629  1006 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.287948   881 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:25.288419   985 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:25.288682   881 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:25.282454 29051 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:25.286443  1001 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:25.285234 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:25.264005   903 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:25.235958   994 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.164784   828 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.282680 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:25.280790   986 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:25.279815   967 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.735s	user 0.005s	sys 0.007s
I20260709 13:12:25.304236 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:25.305488 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.315649 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:25.316157 29712 raft_consensus.cc:2410] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:12:25.306102 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.307511 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.309890  1005 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.310513  1004 raft_consensus.cc:2764] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:12:25.320257 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.311961 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:25.312101 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.322435 29083 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:25.311973 29709 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:12:25.275707   987 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:25.313886 29713 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:12:25.317009 29714 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:25.315094 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.234942   991 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.317713   967 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:25.319197 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.319846 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.334781 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:25.319751 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.288081   955 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:25.322584 29708 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:25.335568 29003 raft_consensus.cc:2393] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:25.324319 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.337199 29715 raft_consensus.cc:2368] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:25.296381   837 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.297178   943 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:25.326967 29717 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:25.317988  1007 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.141s
I20260709 13:12:25.297791   937 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:25.266215   523 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.299582   903 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.334937 29009 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 10: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:25.287436   897 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:25.315763   933 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.340530 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.344558 29009 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:25.279891   874 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.317826  1003 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:25.345394 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.285298   811 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:12:25.355830 29006 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:25.330147   826 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.329418  1019 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:25.285689   964 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.360267 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:25.361409 29084 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:25.360471 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.270030   996 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.341511   943 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.346086   903 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:25.365630 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:25.364274   826 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.342119   822 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:12:25.372776  1020 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:25.370906 29004 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:25.375326 29084 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:25.385020 29711 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:25.348654   837 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.392866 29084 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:25.374979   999 log.cc:927] Time spent T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:12:25.394495  1017 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.375865   943 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:25.336261  1005 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.400372  1005 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:25.400758 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.371644  1003 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.401877 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.402452 29009 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:25.403115 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.403470 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:25.403680 29006 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:25.404503 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.404573 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:25.405108 29009 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:25.405088 29006 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:25.406244 29006 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:25.402483  1003 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.400794 29083 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:25.410086 29083 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:12:25.352181   937 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.411872 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.412297 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.424721 29083 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:12:25.407537  1003 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.408466  1019 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:25.376538   987 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:25.438009 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.442440 29083 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:25.413726  1006 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.421150  1028 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:25.452399  1009 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:12:25.455333 29083 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:12:25.441346  1019 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.407521   943 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.445703  1002 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.463307 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2505) took 1030 ms. Trace:
I20260709 13:12:25.466848 29009 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:25.435451   875 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.443513  1003 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.468142  1023 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:25.459503   958 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.448185   704 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.464460 29001 rpcz_store.cc:276] 0709 13:12:24.432827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.774066 (+341239us) service_pool.cc:224] Handling call
0709 13:12:24.774411 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:24.780558 (+  6147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:24.780593 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:24.792677 (+ 12084us) log.cc:844] Serialized 26 byte log entry
0709 13:12:24.892287 (+ 99610us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:24.892335 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.892426 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:25.463105 (+570679us) raft_consensus.cc:1685] finished
0709 13:12:25.463128 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.463252 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11745,"raft.queue_time_us":6307,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":6322,"thread_start_us":78803,"threads_started":1,"wal-append.queue_time_us":228649}
I20260709 13:12:25.480294   874 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:25.481632   919 consensus_meta.cc:287] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.817s	user 0.000s	sys 0.009s
I20260709 13:12:25.482020   919 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.482952 29082 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
W20260709 13:12:25.484163   849 consensus_meta.cc:287] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.820s	user 0.011s	sys 0.000s
W20260709 13:12:25.484262  1010 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:12:25.484591   849 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.485370   937 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.485483   996 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.458855  1019 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:25.394971  1015 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.122s
I20260709 13:12:25.486480 29003 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:25.490883 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.462847  1028 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:25.504325  1016 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:12:25.525118 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:25.525766  1018 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:12:25.529881   992 consensus_meta.cc:287] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.002s
I20260709 13:12:25.516553   964 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.530587   992 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:25.537078   875 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.538781   903 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.427748  1017 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.525168   704 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.541934   849 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:25.545077   932 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.535s	user 0.002s	sys 0.008s
I20260709 13:12:25.553131  1017 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.560698   817 raft_consensus.cc:3055] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:25.561199   817 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:25.562040   817 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.563104   817 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:25.565735  1022 log.cc:927] Time spent T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:12:25.567655  1003 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.569304   943 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.540666   837 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.522502   919 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.571007 29076 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.521646   994 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.573529 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:25.574630 29051 leader_election.cc:400] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:25.575001 29051 leader_election.cc:403] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:25.574735 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.582059 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.582873 29009 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:25.586905   971 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:25.593928 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:25.587280  1013 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.587791   992 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:25.595944   971 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.616808   971 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:25.621413  1001 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.622709   780 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.625030 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:25.626286   960 consensus_meta.cc:287] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.812s	user 0.013s	sys 0.000s
I20260709 13:12:25.596936   892 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:25.626766   994 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.626914   960 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.627364   994 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.628983   972 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.629477   960 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.629742   994 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.631124   943 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:25.631637 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2507) took 1198 ms. Trace:
I20260709 13:12:25.631830 29081 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:25.631654   943 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.634924   943 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.635751 29005 rpcz_store.cc:276] 0709 13:12:24.433301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.747625 (+314324us) service_pool.cc:224] Handling call
0709 13:12:25.631442 (+883817us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400139554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:25.631579 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397133184}
I20260709 13:12:25.638171 29122 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:25.638840 29151 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1789) took 1256 ms (client timeout 1527 ms). Trace:
W20260709 13:12:25.639298 29151 rpcz_store.cc:269] 0709 13:12:24.382043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.418864 (+ 36821us) service_pool.cc:224] Handling call
0709 13:12:25.638688 (+1219824us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54001cfcd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:25.638789 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210844544}
I20260709 13:12:25.650236 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:25.627014   892 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.635141  1019 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.640496  1001 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.605847  1013 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.662267  1028 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.652176   973 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:25.655158   970 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.653682   892 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:25.652071   932 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:12:25.677488   932 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:25.677896   932 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:25.678726   932 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.609828   897 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.670392 29715 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:25.682130 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:25.683692 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.664203   903 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:25.684417 28975 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.527s (SENT) [suppressed 11 similar messages]
I20260709 13:12:25.684520   960 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:25.671931   973 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:25.679984   932 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:25.688992 29713 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:25.693527  1019 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.694085   960 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:25.710253   960 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:25.718734  1029 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:12:25.720335 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2506) took 1287 ms. Trace:
I20260709 13:12:25.721175   993 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:25.721717   993 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:25.720702 29007 rpcz_store.cc:276] 0709 13:12:24.433045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.789820 (+356775us) service_pool.cc:224] Handling call
0709 13:12:24.804437 (+ 14617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:24.808519 (+  4082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:24.808552 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:24.812761 (+  4209us) log.cc:844] Serialized 26 byte log entry
0709 13:12:25.453683 (+640922us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b180042e680. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:12:25.488286 (+ 34603us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:25.495408 (+  7122us) log.cc:844] Serialized 14 byte log entry
0709 13:12:25.495615 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.495686 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:25.720052 (+224366us) raft_consensus.cc:1685] finished
0709 13:12:25.720149 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.720280 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9025,"prepare.queue_time_us":7374,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":6867,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":6451,"spinlock_wait_cycles":21226240,"thread_start_us":34433,"threads_started":1,"wal-append.queue_time_us":699223}
I20260709 13:12:25.695127   973 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:25.694960   992 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:25.739542   897 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.742728   987 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:25.743340   987 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.745513  1013 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.746593  1005 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:25.748633 29076 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:25.749133 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:25.749790 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:25.750340 29078 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:25.750279   973 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.750880 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.746908   892 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.747123  1005 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.752671   993 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.12, Last appended by leader: 12, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.754009  1005 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.754155   993 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:25.754492   892 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.754627 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:25.755394 29709 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 7: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:25.755637 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.756790 29712 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [9.12-9.12]
I20260709 13:12:25.758073 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:25.758610 29008 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:25.758914 29078 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: [9.7-12.9]
W20260709 13:12:25.760164 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 75778.4us
I20260709 13:12:25.768839 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.771050 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:25.751973  1033 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:12:25.777727 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.782729 29008 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:25.745551   987 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.786840 29708 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:25.766963  1013 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:25.770403   907 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:25.769057   576 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:25.794844 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:25.769573  1019 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:12:25.770169   817 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:12:25.794211   907 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:25.797559  1013 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:25.816504  1035 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:12:25.816282  1013 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.816023   907 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.820279   907 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:25.823400   973 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.818405   817 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.831074   817 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.804021   997 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:12:25.831948   828 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:25.833807   817 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.801915   576 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:25.835472   881 consensus_meta.cc:287] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.003s
I20260709 13:12:25.916925   997 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.834748   971 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.919310   881 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.950281 29084 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:25.937856   997 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.955292   576 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.983431   960 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.984007   960 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.984422 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:25.986449   960 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.958020   828 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.986833   523 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.987089   828 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:25.988410 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:25.989070 29076 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:25.989198   828 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:25.958506 29005 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
W20260709 13:12:25.961944 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.964416   995 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:25.990818 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:25.991595 29005 raft_consensus.cc:2368] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
I20260709 13:12:25.973587 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.973135 29002 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:12:25.993255 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:25.993630   997 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.994060 29006 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:25.973905 29003 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:25.994556   683 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.976562 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:25.995424 29009 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:25.995738   881 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.996465 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:25.984463 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:25.997608 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:25.991734   971 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:25.998471 29125 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:25.999907   852 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.001094   902 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:25.975425 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:25.994472 29680 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:26.002060   986 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.002413 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:26.002933 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.977185 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:26.003165   783 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:26.003718 29010 raft_consensus.cc:2368] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
I20260709 13:12:26.003990   783 raft_consensus.cc:697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:26.004177   852 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:26.003233   961 raft_consensus.cc:2804] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:25.980082   923 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:26.005193   961 raft_consensus.cc:697] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:26.005486   982 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.005640   923 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:26.005039   783 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:25.990677   995 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.006137   923 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:26.006179   961 consensus_queue.cc:237] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.007107   923 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.008370   970 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:26.008878   970 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:26.008991   836 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.009275 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:26.010020   990 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:26.009578   836 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:26.010324 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.010532   990 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.011963   836 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.012506   990 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.012650   960 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:26.002552 29711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 527) took 3252 ms (client timeout 1648 ms). Trace:
I20260709 13:12:26.013206   960 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:26.013341 29711 rpcz_store.cc:269] 0709 13:12:22.749632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.749735 (+   103us) service_pool.cc:224] Handling call
0709 13:12:26.002372 (+3252637us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b54005f3554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:26.002478 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5933376384}
I20260709 13:12:26.012528   836 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:26.015707   836 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.016464 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.017864   836 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.018126   530 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:26.008989 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2513) took 1032 ms. Trace:
I20260709 13:12:26.018682 29010 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:12:26.019438 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:26.019124 29159 rpcz_store.cc:276] 0709 13:12:24.976200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.977853 (+  1653us) service_pool.cc:224] Handling call
0709 13:12:24.978144 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:24.992523 (+ 14379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:24.992554 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:24.997276 (+  4722us) log.cc:844] Serialized 26 byte log entry
0709 13:12:25.147086 (+149810us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:25.147126 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.008610 (+861484us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b54003f6bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:26.008654 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.008701 (+    47us) raft_consensus.cc:1685] finished
0709 13:12:26.008716 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.008915 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12473,"raft.queue_time_us":16775,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":4480,"spinlock_wait_cycles":3483904,"thread_start_us":139819,"threads_started":1,"wal-append.queue_time_us":203976}
I20260709 13:12:26.007555   995 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.022235 29711 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:26.022565 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.022831 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:26.023335 29077 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:26.023411 29005 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.023926   826 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:26.025882 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.004633   852 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:26.026229 32570 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:26.026602   852 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:26.027016 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.009692   970 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.027626 29081 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:26.028196 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.028360 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.028821 29077 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:26.028869 29082 raft_consensus.cc:3060] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:26.029773 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.030476 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.031481 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.031958 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:26.032418 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.032959 29084 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:26.034365 29077 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:26.037225 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 61867.3us
I20260709 13:12:26.042912 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:26.043633 29683 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:26.047390 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.055903 29079 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:26.056758 29081 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:26.057272 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.059274 29079 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:12:26.061070 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.065268 29081 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:26.065296 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:26.068660 29003 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:26.068845 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:26.070104 29008 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:26.032534 29006 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:26.070909 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.073637 29081 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:26.077005 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 44634.2us
I20260709 13:12:26.078272 29009 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:26.081698 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:26.137969 29717 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:26.083169 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.084424 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.139250 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.139875 29077 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:26.140663 29708 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:26.141006 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:26.141911 29007 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:26.103513 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:26.103237 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.143971 29711 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:26.132257 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:26.127686 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:26.134147 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:26.146090 29008 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:26.132524 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:26.151744 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.153357 29010 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:12:26.105405 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.155441 29083 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 10: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:26.125903 29084 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:26.102849 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:26.164570 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.186474 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:26.186592 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.187192 29078 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:26.187788 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:26.175283 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.188524 29084 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:26.132688   902 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.165793 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.189932 29077 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:26.190192 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:26.156286 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:26.184144 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80715us
I20260709 13:12:26.172605   982 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:26.172907 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:26.164746 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61947.4us
I20260709 13:12:26.187984 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.192770 29007 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:12:26.084115   836 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:26.083921   889 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:26.179505  1030 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.559s	user 0.000s	sys 0.002s
I20260709 13:12:26.165275 29006 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:26.104766   683 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:26.176343 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.065644   881 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:26.104728   995 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:12:26.132870   993 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:26.140944   669 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:26.104943   992 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:12:26.194832 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:26.204927 29003 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:26.186565   780 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:26.194639 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:26.196483 29708 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:26.211122 29083 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:26.199074 29078 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.227602 29048 leader_election.cc:400] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
I20260709 13:12:26.075802   909 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 13:12:26.199479 29005 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:26.197289 29008 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:26.215085 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:26.212304 29079 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:26.238245 29002 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.196246 29713 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:26.201332 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.198740 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:26.165671   960 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.225304 29683 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:26.205442 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.199117 29009 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:26.248411 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.248788 29006 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:26.228686 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:26.234193 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:26.236373 29050 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:26.245636 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46966.9us
I20260709 13:12:26.250219 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.251204 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:26.225996   838 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:26.249488 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:26.084296   997 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.134s
I20260709 13:12:26.252866 29125 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:26.254180 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.258447 29084 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:26.254585 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:26.259121 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.066154   998 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:12:26.217742   881 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.197222   982 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.252272 29003 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
W20260709 13:12:26.296650 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 60341.5us
I20260709 13:12:26.226322 32630 raft_consensus.cc:3055] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:26.260264 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:26.198626  1015 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:12:26.301939 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:26.309283 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.307322 29084 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:26.326880 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1627) took 1198 ms. Trace:
I20260709 13:12:26.332337 29079 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:26.332135 29085 rpcz_store.cc:276] 0709 13:12:25.128152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.144380 (+ 16228us) service_pool.cc:224] Handling call
0709 13:12:25.144669 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:25.158173 (+ 13504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:25.158218 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:25.177540 (+ 19322us) log.cc:844] Serialized 26 byte log entry
0709 13:12:25.496593 (+319053us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:25.496645 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.496715 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.178974 (+682259us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:26.261416 (+ 82442us) raft_consensus.cc:1685] finished
0709 13:12:26.261444 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.326792 (+ 65348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35446,"raft.queue_time_us":20405,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":8018,"spinlock_wait_cycles":39987456,"thread_start_us":259801,"threads_started":1,"wal-append.queue_time_us":381143}
I20260709 13:12:26.334868 29084 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:12:26.336211 29084 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:26.322365 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:26.293495 29125 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:26.337694 29084 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [9.5-10.6]
I20260709 13:12:26.308248 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:26.226859   993 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.228758   934 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 13:12:26.250334   777 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:26.341931 29077 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:26.255518   889 raft_consensus.cc:2804] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:26.339035 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:26.340242 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46646.4us
I20260709 13:12:26.363498   777 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.321272 29002 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:26.309254 32630 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:26.390143 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:26.200428   902 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.365392 29078 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:26.376564 29077 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:26.376859 29079 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:26.377300 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:26.223759   683 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.362953   922 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:26.380344 29085 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:26.229642   669 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.297237   995 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:26.377635 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:26.258941   990 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:26.389288 29004 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:26.405918 29006 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:26.312028   982 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.300076   881 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:26.292735   836 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:26.405450 29078 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:26.299619   960 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:12:26.252607   986 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:26.308215   932 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:26.419358 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:26.301141   703 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:26.391978   997 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:26.384613   998 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.436918 29708 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:26.391844   777 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:26.414081   922 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.410089   889 raft_consensus.cc:697] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:26.437613 29083 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
W20260709 13:12:26.440081 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.450157 29081 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:26.416841   995 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.427162   828 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:26.460345   828 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.460840   828 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.459210   669 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.486853 29082 raft_consensus.cc:2468] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:26.489668 29717 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:26.435791   986 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.496109 29158 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:12:26.498961 29158 pending_rounds.cc:85] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 12
I20260709 13:12:26.499253 29158 pending_rounds.cc:107] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:12:26.499539 29158 raft_consensus.cc:2889] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:26.468600   889 consensus_queue.cc:237] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.435225   969 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:26.454430  1052 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:12:26.502156 32570 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:26.446446   836 raft_consensus.cc:2764] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:12:26.447362   703 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.451097   837 raft_consensus.cc:2764] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:12:26.454555   922 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.443619   992 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:12:26.453138   932 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:26.482964   995 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:26.516603  1047 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:12:26.453912   998 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.523123   993 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.510901   969 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.581629   967 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:26.582170   967 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:26.586181 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2512) took 1887 ms. Trace:
I20260709 13:12:26.586678 29158 rpcz_store.cc:276] 0709 13:12:24.698363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.699572 (+  1209us) service_pool.cc:224] Handling call
0709 13:12:24.699846 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.500032 (+1800186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.585834 (+ 85802us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b540041a5d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:26.586103 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4129883008}
I20260709 13:12:26.587869   783 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:26.583194   967 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.446525   990 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:12:26.502314   986 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.501261   669 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:26.550637 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:26.452013   997 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.511929   703 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:26.496929 32630 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.572386  1013 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.571269   911 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:26.505824 32570 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.473397   852 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.578842   971 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:26.635283 32570 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:26.645509   998 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.647235   969 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:26.651252  1051 log.cc:927] Time spent T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:12:26.599377 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 343488us
I20260709 13:12:26.592417   783 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:26.664675 29048 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:26.634326   967 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:26.571475   932 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.675282 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:26.634635   971 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.639456 32630 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:26.643456   828 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.679833  1015 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:26.704496 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 402624us
I20260709 13:12:26.611541  1013 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:26.711522 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 330572us
I20260709 13:12:26.680585   971 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.678012   836 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:26.711063 29051 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:26.676304   912 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:12:26.702422   969 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.703105   683 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.701738 29048 leader_election.cc:400] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
I20260709 13:12:26.702764  1015 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:26.720961 29048 leader_election.cc:403] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:12:26.678650   902 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:26.760171 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.762609 29122 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:26.760458  1015 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:26.765350   998 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:26.765787   998 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:26.766168   998 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:26.724524   969 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.770241   969 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.774747   520 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:26.774363   998 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.775156   520 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:26.764447 29122 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:26.776072   520 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.776677   969 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:26.777120   969 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.777237   520 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:26.777498   969 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:26.781431  1050 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 13:12:26.781433  1013 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.785284 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:26.813553 32570 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:26.784233   988 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:12:26.761279   989 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:26.819003 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 399667us
I20260709 13:12:26.819936 32570 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:26.806612   683 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:26.821062 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:26.809341   960 raft_consensus.cc:2764] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:12:26.830032 32570 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:26.842202   530 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.824s	user 0.011s	sys 0.000s
I20260709 13:12:26.842969   934 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.843465   934 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:26.846351 28906 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51350 (request call id 149) took 35569 ms (client timeout 15000 ms). Trace:
I20260709 13:12:26.850003 28905 catalog_manager.cc:5697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:26.851509   777 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.853650   777 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.851490 28905 catalog_manager.cc:5697] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 6 to 10, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
W20260709 13:12:26.854842   780 consensus_meta.cc:287] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.000s
W20260709 13:12:26.829389 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.895547 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:26.855612 28905 catalog_manager.cc:5697] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:26.856220   992 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
W20260709 13:12:26.846868 28906 rpcz_store.cc:269] 0709 13:11:51.311271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.311463 (+   192us) service_pool.cc:224] Handling call
0709 13:12:26.846263 (+35534800us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:21.897199 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:21.898206 (+  1007us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:21.898835 (+   629us) write_op.cc:435] Acquired schema lock
0709 13:12:21.899951 (+  1116us) tablet.cc:662] Decoding operations
0709 13:12:21.992973 (+ 93022us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:21.993066 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:21.993108 (+    42us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:12:21.994234 (+  1126us) tablet.cc:701] Row locks acquired
0709 13:12:21.994290 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:12:21.994507 (+   217us) write_op.cc:270] Start()
0709 13:12:21.994653 (+   146us) write_op.cc:276] Timestamp: P: 1783602741994428 usec, L: 0
0709 13:12:21.994689 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:12:21.995545 (+   856us) log.cc:844] Serialized 3954 byte log entry
0709 13:12:22.009540 (+ 13995us) op_driver.cc:464] REPLICATION: finished
0709 13:12:22.018812 (+  9272us) write_op.cc:301] APPLY: starting
0709 13:12:22.087705 (+ 68893us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:22.872777 (+785072us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:22.872917 (+   140us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:22.930006 (+ 57089us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:22.930302 (+   296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:12:22.930344 (+    42us) write_op.cc:312] APPLY: finished
0709 13:12:22.930675 (+   331us) log.cc:844] Serialized 97 byte log entry
0709 13:12:22.930897 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:22.931020 (+   123us) write_op.cc:454] Released schema lock
0709 13:12:22.931083 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:22.931536 (+   453us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:22.931565 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1338871,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":9011,"apply.run_cpu_time_us":30320,"apply.run_wall_time_us":912642,"cfile_cache_hit":38,"cfile_cache_hit_bytes":39710,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6217,"cfile_init":1,"lbm_read_time_us":53101,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":3,"lbm_reads_lt_1ms":4,"num_ops":10,"prepare.queue_time_us":616427,"prepare.run_cpu_time_us":5713,"prepare.run_wall_time_us":101283,"replication_time_us":14679,"thread_start_us":35038,"threads_started":2}]]}
I20260709 13:12:26.720925   783 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.830219 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:26.905143 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 75000.9us
W20260709 13:12:26.884677 29123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.013s	sys 0.027s
W20260709 13:12:26.906834 29123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.013s	sys 0.027s
W20260709 13:12:26.907287 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.887398 32570 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:26.889624 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 513577us
I20260709 13:12:26.903661 28905 catalog_manager.cc:5697] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 3 to 8, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:26.911113 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.912154 28905 catalog_manager.cc:5697] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:26.913458 28905 catalog_manager.cc:5697] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:26.908608 32570 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.917141 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.915138 28905 catalog_manager.cc:5697] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 5 to 9, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 9 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:26.919639 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.905695 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:26.921149 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.922693 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.906308 29122 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:26.919107 28905 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 7 to 9, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 9 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.921860 29714 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:26.925320 28905 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 8 to 11, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 11 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.882313   997 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.826985   989 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.922084   683 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:26.952174   669 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.952623   683 raft_consensus.cc:697] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:26.952765   669 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:26.953163 28905 catalog_manager.cc:5697] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 6 to 12, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 12 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:26.953528   683 consensus_queue.cc:237] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.954898   669 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:26.954622 28905 catalog_manager.cc:5697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 8 to 10, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 10 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:26.905334   780 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:26.933024   828 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.929327 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.940385   902 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:26.959710   902 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:26.960245 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:26.960711 29717 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:26.961731   902 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:26.926090   934 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.957674   989 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:26.962625 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.009159 29150 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.009536 28905 catalog_manager.cc:5697] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 8 to 11, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:26.928260 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:26.962575   828 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:27.023124 29150 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:26.903048   530 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:26.940909 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.904692   982 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:27.050747 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.964082 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.026574 29155 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 12. Preceding OpId from leader: term: 10 index: 12. (term mismatch)
I20260709 13:12:27.051393 29155 pending_rounds.cc:85] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 11
I20260709 13:12:27.050998 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:26.944892 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:27.029181 29156 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:27.031540 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:27.014489   923 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.017s	sys 0.043s
I20260709 13:12:26.895774   852 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.052806 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.053109 29152 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:26.949203  1015 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:27.024152   907 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.019s	sys 0.043s
I20260709 13:12:27.012394   967 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.053392 29159 raft_consensus.cc:3060] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:27.053802   967 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.055074  1015 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.056099   967 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.057303   967 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:27.057785   967 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:27.058692   967 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.060075 29159 raft_consensus.cc:2468] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
W20260709 13:12:27.060405   907 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.020s	sys 0.043s
I20260709 13:12:26.942171   996 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:26.962539   934 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:12:27.041742 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.062810 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:27.063100 29154 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:27.063539 29157 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:27.064144   537 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.961501   971 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.051658 29155 pending_rounds.cc:107] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 9.12
W20260709 13:12:26.929180 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:27.033643 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.075707 29155 raft_consensus.cc:2889] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:27.076167 29151 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:26.920423  1013 consensus_queue.cc:1059] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
W20260709 13:12:27.061470 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 116697us
W20260709 13:12:27.078852 28895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 149719us
I20260709 13:12:27.079107 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:26.941272   909 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.083552 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.096256 29122 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:27.097594 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:27.097754 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:27.074535   537 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:27.064091   881 consensus_meta.cc:287] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.000s
I20260709 13:12:27.063061 29153 raft_consensus.cc:2955] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 3 to 7, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:27.028998   995 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:27.117758 29049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.021s	sys 0.018s
W20260709 13:12:27.118090 29049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.021s	sys 0.019s
W20260709 13:12:27.119788 29157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2503) took 3006 ms (client timeout 2189 ms). Trace:
I20260709 13:12:27.044697 28905 catalog_manager.cc:5697] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 7 to 10, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 10 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.077801 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.074527 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.061173   990 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.071123   988 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:27.071332   902 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:27.114219   998 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:27.141584   902 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.141392   990 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.115264   669 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:27.142071   902 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:27.142359   669 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:27.142767   669 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.143690   990 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:27.107177 29156 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:12:27.062207   923 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.018s	sys 0.043s
W20260709 13:12:27.124136 29157 rpcz_store.cc:269] 0709 13:12:24.113517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.113654 (+   137us) service_pool.cc:224] Handling call
0709 13:12:27.119570 (+3005916us) spinlock_profiling.cc:243] Waited 2.92 s on lock 0x7b5400055d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:27.119710 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6712313216}
I20260709 13:12:27.125458 29152 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:27.153527 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.156177  1015 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.156682  1015 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.158835   902 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.159669 29082 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:27.126437 28905 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 9 to 11, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.068022  1060 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.160533  1060 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.161250 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.162442 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.162602  1060 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:27.144307 29151 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:27.162978 29083 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:27.144318  1061 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:12:27.141429   988 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.163975   988 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.164651 29078 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:12:27.166740 29078 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:27.167290 29155 raft_consensus.cc:2368] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:27.145524   982 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.146009  1058 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:27.168347   995 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:27.095266   911 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:12:27.113919  1063 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:27.170305   971 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.170629  1063 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:27.113548   997 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:27.125620   881 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:27.171633   922 consensus_meta.cc:287] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.611s	user 0.008s	sys 0.002s
I20260709 13:12:27.171537  1063 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.172183   922 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.172817  1063 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.173305 29080 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:12:27.102909   520 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:27.078460   996 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:27.174724   922 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:27.125914   703 consensus_meta.cc:287] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 13:12:27.175959  1059 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:27.105310   986 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.001s
I20260709 13:12:27.176916  1059 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.177054   986 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.113413   970 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:12:27.182835   969 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.113296   993 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:27.064229   530 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.184363 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.185710   530 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.186167   969 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.175935   703 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.188162 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:27.174794   996 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.179759   986 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.188192 29150 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:27.188576 29159 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:27.195948 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.200276 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.195623 28905 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.187229 29083 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:27.202135 28905 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:12:27.203846 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 400897us
I20260709 13:12:27.204689 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.205551 29157 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
W20260709 13:12:27.205648 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 504060us
I20260709 13:12:27.206295 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:27.208926   530 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:27.209625   530 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:27.211073 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.172890   881 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.113018   909 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.201510   826 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:27.158913  1015 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.223897 29152 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:27.231086 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.231801 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:27.203392 28905 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.232580 29715 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:27.232245 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.210995  1057 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.204365   993 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:27.236460 28905 catalog_manager.cc:5697] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 1 to 9. New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.237241 29158 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.238423 28905 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:27.223084   780 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:12:27.228080 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 322431us
I20260709 13:12:27.255419 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.256496 29157 raft_consensus.cc:2368] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:27.256623 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.228519   826 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.234014 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.243600 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2110) took 1070 ms. Trace:
I20260709 13:12:27.216250  1059 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.204238   523 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:12:27.266146 29080 rpcz_store.cc:276] 0709 13:12:26.173590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.343193 (+169603us) service_pool.cc:224] Handling call
0709 13:12:26.411040 (+ 67847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:27.173666 (+762626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:27.243284 (+ 69618us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400199e54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:27.243527 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752459136}
I20260709 13:12:27.259625 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.267505 29085 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:27.257231 29155 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:27.269093 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.270660 29155 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:27.261049 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.272636 29084 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:27.262689 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.264807 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.275377 29004 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:27.285343 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.286090 29083 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:27.285362 29157 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:27.287168 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.287812 29077 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:27.288651 29152 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:27.289726 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.291252 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.291432 29158 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:27.238440   912 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:27.304432 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.292747 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.305228 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.305665 29079 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:27.306313 29152 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:27.293260 29085 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:27.240025   790 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:27.276561   826 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:27.294430 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.327518 29001 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:27.328446 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.326617 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.329821 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.330605 29001 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
I20260709 13:12:27.331545 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.291782 29004 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:27.332669 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.295895 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.333756 29002 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:27.296646   669 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.296955 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.298355 29009 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:12:27.298489 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.299870   902 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:27.300338 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.341009 29006 raft_consensus.cc:2368] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
I20260709 13:12:27.292351  1064 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:12:27.239791 28905 catalog_manager.cc:5697] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 6 to 9, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 9 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.255035   986 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:27.304586 29084 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:27.276674 32630 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.319079 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.293538   780 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:27.363523 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.363473 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.364439 29002 raft_consensus.cc:2368] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:27.366169 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.366480 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.334982 29003 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:27.329356 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:27.353502 28905 catalog_manager.cc:5697] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 3 to 7, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 7 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:27.364444 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 163405us
I20260709 13:12:27.366725 29157 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.265684   971 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:27.240671  1057 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.370972 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.377568 29006 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:12:27.379778 29004 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:27.380822 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.372686 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:27.367995 28905 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.383430 28905 catalog_manager.cc:5697] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.384604 28905 catalog_manager.cc:5697] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.385949 28905 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.387602 28905 catalog_manager.cc:5697] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
W20260709 13:12:27.389870 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 290191us
I20260709 13:12:27.390137 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.338476 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.388470 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.391129 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:27.391929 29002 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:27.392143 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:27.401499 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:27.402155 29708 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:27.325464   912 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.405993 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.407730 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.340570   790 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.410524 29006 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:27.411404 29155 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:27.412405 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:27.413265 29715 raft_consensus.cc:2410] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:12:27.413647 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.414273 29002 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:27.414355 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.387781 29079 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:27.416234 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.360641   986 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.371465 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.393430   907 raft_consensus.cc:3055] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:27.362466 32630 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.407899 28905 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:27.424233 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.425526 29079 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:27.389261  1076 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:27.424180 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.428110 29153 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:27.429735 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.430572 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.431427 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.431574 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.441207 29157 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:27.436797 29009 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
W20260709 13:12:27.437769 28975 leader_election.cc:336] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Timed out: RequestConsensusVote RPC to 127.27.63.67:40957 timed out after 1.527s (SENT)
I20260709 13:12:27.437391 28905 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 7 to 11, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 11 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:27.415719  1069 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.420466   982 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:27.448344   982 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.448411 29003 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:27.391196   971 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:27.439986 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 123573us
I20260709 13:12:27.412971  1075 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:27.458510 29006 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:27.382009 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 184017us
I20260709 13:12:27.459841 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:27.462611 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:27.418929 29152 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:27.446830 28975 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:27.444801 29004 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:27.439491 28976 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:27.444954 29155 raft_consensus.cc:2376] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11. Re-sending same reply.
I20260709 13:12:27.422609   907 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:27.415828   902 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:12:27.445770 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.447444 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.466343 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:27.451824  1067 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:12:27.448993   982 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:27.451542   909 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.381613   993 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.430014  1076 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.409421   988 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.450459 28905 catalog_manager.cc:5697] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:27.455773 32630 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.466226 29002 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:27.478403 29157 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:27.478682 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:27.465323 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.482331 29083 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:27.470984   828 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:27.484777 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.509593 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:27.526264 29714 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.513285 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.527791 29002 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:27.487435 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:27.514808 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.451948   669 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.454487  1057 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.482191   530 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.477536 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.544732 29153 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:27.474432   934 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:27.526130 29079 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
W20260709 13:12:27.552654  1055 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.688s	user 0.006s	sys 0.000s
I20260709 13:12:27.552802 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.553612 29079 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:27.554380 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.554658 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2351) took 1402 ms. Trace:
I20260709 13:12:27.555485 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.556988 29079 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 14: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:27.557744 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.558434 29079 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:27.559162 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.559835 29079 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:27.560362 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.560557   967 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:27.560948 29153 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:12:27.561414 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.561123   967 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.561981 29083 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.562500   934 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:27.563346   934 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.521546  1082 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:27.564472  1082 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:27.565505   986 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.565279  1082 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.509426 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.456306  1063 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:12:27.517446 29158 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:27.514101 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.508570   988 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.538602 29002 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:27.726473 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.521533 29152 raft_consensus.cc:2410] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:12:27.540208 29006 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:12:27.492615 29082 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:27.713698 29051 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:27.729869 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.491902 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.544544 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.500573 29080 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:12:27.561908 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.733528 29153 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:27.734229 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.734936 29153 raft_consensus.cc:2410] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:27.566916 29159 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:27.555083 29010 rpcz_store.cc:276] 0709 13:12:26.152470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.154199 (+  1729us) service_pool.cc:224] Handling call
0709 13:12:26.186377 (+ 32178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:26.186615 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.186655 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:26.187129 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:12:26.605629 (+418500us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:26.605672 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.605743 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:27.552102 (+946359us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:27.554370 (+  2268us) raft_consensus.cc:1685] finished
0709 13:12:27.554392 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:27.554602 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31918848,"thread_start_us":380044,"threads_started":1,"wal-append.queue_time_us":634744}
I20260709 13:12:27.736891 29002 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:12:27.568099 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.739217 29151 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:27.571308 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.747912 29077 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:27.494066 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:27.516718 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:27.493122 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:27.749816 29003 raft_consensus.cc:2393] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:27.516983   902 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:27.436600   780 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.510507   919 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:27.586530 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:27.770830 29079 raft_consensus.cc:2393] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:27.581033 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:27.703990 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.500887  1068 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.455964  1069 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.781929 29715 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:27.723361  1057 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:27.589037 29714 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:27.509593   790 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.508409   703 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:27.517028 29154 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:12:27.714900 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 137103us
I20260709 13:12:27.715929 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.790295 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.722517 29083 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:27.510821  1076 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:27.566336   783 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.745720 29680 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:27.752826 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.793432 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:27.793963 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 277294us
I20260709 13:12:27.794747 29153 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:27.766438 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.706370   932 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.772219 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.777614 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.796953 29150 raft_consensus.cc:2410] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:27.796873 29155 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
W20260709 13:12:27.779855 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 48863.3us
I20260709 13:12:27.710628   967 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:27.781651 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 200817us
I20260709 13:12:27.730593  1082 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:27.587177  1051 log.cc:927] Time spent T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:12:27.763038  1090 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:12:27.474241  1013 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:12:27.759603 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.513795  1062 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.203s
I20260709 13:12:27.794876 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:27.758265  1070 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:27.760594 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.805047 28975 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:27.865859 29152 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:27.866189 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61291.7us
W20260709 13:12:27.808034 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 251440us
I20260709 13:12:27.868474 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:27.808034 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 62414.3us
I20260709 13:12:27.869799 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:27.811088 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.812651 29153 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:12:27.814937 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:27.815310 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.816759 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:27.817195 32570 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.877142 32570 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:27.802883 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 334283us
I20260709 13:12:27.861302 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:27.878902 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:27.586359   907 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:27.823491 29082 raft_consensus.cc:2393] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:27.770455   919 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:27.730161  1015 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
W20260709 13:12:27.855168 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 65245.3us
I20260709 13:12:27.890103 29050 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:27.791551  1063 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:27.885155 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:27.891423 29157 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:27.880949 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:27.887025 28975 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:27.893810 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 99012.1us
I20260709 13:12:27.894071 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:27.894572 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:27.896179 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:27.888813 29006 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
I20260709 13:12:27.800419  1074 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:27.816222   988 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:12:27.911418 29708 consensus_meta.cc:287] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 13:12:27.899837 29050 leader_election.cc:400] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
W20260709 13:12:27.878923  1079 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:12:27.912201 29050 leader_election.cc:403] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:27.913350 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:27.914551   902 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:27.918187 29006 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:27.914762  1062 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:27.858974   967 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:27.886216   826 consensus_meta.cc:287] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.000s
I20260709 13:12:27.912235 29708 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:27.864526  1070 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.928360 29050 leader_election.cc:400] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
W20260709 13:12:27.914165 29742 heartbeater.cc:646] Failed to heartbeat to 127.27.63.126:43467 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.27.63.126:43467 timed out after 15.000s (SENT)
I20260709 13:12:27.808946  1057 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.918104   988 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:27.934218 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 464223us
I20260709 13:12:27.940479 29151 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:27.945696 29077 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:27.947741 29152 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:27.811182   932 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.997529  1069 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:27.998399   932 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:27.998972   932 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:27.974639 29050 leader_election.cc:403] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:28.000102 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 71977.5us
I20260709 13:12:28.000756 29050 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:27.977728 29157 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:27.787118  1068 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.003684  1068 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:27.916574   907 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.961856   912 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:28.004323 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:28.004997   912 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:28.005738   912 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:28.006420   912 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:28.006718 29006 raft_consensus.cc:1275] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:28.006788   912 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.007405   912 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:27.931782   777 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:28.008332   777 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:28.008514   988 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:28.008863   777 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.985605  1070 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.973471   967 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:27.927076   881 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:12:27.910133   996 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:27.916993  1061 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:27.886289   530 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:28.038998  1103 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:28.018000  1077 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.547s	user 0.001s	sys 0.000s
I20260709 13:12:27.925532   986 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:27.993717 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:28.072352 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 201603us
I20260709 13:12:28.072746 29083 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:27.933516  1063 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:12:27.994544 29081 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:27.983755   790 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:12:27.963349  1075 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:27.988137 29002 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:28.066128 32570 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:27.993188   852 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:28.004189   989 raft_consensus.cc:2764] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:12:27.994364 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:28.019616   703 raft_consensus.cc:2764] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:12:28.127455 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2115) took 1410 ms. Trace:
I20260709 13:12:28.004211  1068 raft_consensus.cc:2764] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:12:28.081633 29083 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:28.106942 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:28.074034   530 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:28.022922  1061 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:28.110136 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:28.110386 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:28.128019 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:27.983629   826 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.072561   986 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.995951   537 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:28.143185 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 281926us
I20260709 13:12:28.127885 29076 rpcz_store.cc:276] 0709 13:12:26.717024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.717174 (+   150us) service_pool.cc:224] Handling call
0709 13:12:26.717605 (+   431us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:12:27.198362 (+480757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:27.198394 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:12:27.206391 (+  7997us) log.cc:844] Serialized 104 byte log entry
0709 13:12:27.598632 (+392241us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:27.724090 (+125458us) log.cc:844] Serialized 14 byte log entry
0709 13:12:28.126897 (+402807us) log.cc:844] Serialized 14 byte log entry
0709 13:12:28.127133 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:28.127204 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:28.127242 (+    38us) raft_consensus.cc:1685] finished
0709 13:12:28.127253 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:28.127402 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63818,"prepare.queue_time_us":125208,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":10234,"raft.queue_time_us":224197,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":15612,"spinlock_wait_cycles":71552,"thread_start_us":1178630,"threads_started":3,"wal-append.queue_time_us":808543}
I20260709 13:12:28.165160 29713 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:28.106290  1063 consensus_queue.cc:1243] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:12:28.168889 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 273667us
I20260709 13:12:28.022804   919 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.135759   530 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:28.172756   934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.005s	sys 0.026s
I20260709 13:12:28.168743  1061 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:28.160748   703 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:28.128010  1069 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:12:28.179401   986 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:28.182816   971 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:28.192176   530 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:28.206827   934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.005s	sys 0.026s
W20260709 13:12:28.223897 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 342979us
W20260709 13:12:28.229300 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 227776us
I20260709 13:12:28.229578 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:28.230569 29050 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:28.230863 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:28.231683 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 337146us
I20260709 13:12:28.232887 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:28.191982   881 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 13:12:28.215853 29001 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.005s	sys 0.012s
I20260709 13:12:28.236135 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:28.237087 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 350139us
I20260709 13:12:28.262290 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.265586 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:28.186326 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:28.269225 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:28.272147 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.273006 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:28.273790 29082 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:28.275218 29076 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:12:28.277360 29001 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.006s	sys 0.013s
I20260709 13:12:28.265969 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.281745 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:28.282208 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 95948.8us
W20260709 13:12:28.295050 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 62244.4us
I20260709 13:12:28.296377 29122 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:28.297135 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 111702us
I20260709 13:12:28.298669 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:28.260202   971 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:28.305671 29708 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:12:28.222744  1112 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:12:28.259899  1069 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.279672  1064 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:28.283324  1056 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.307353 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:28.342624 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 76751.1us
I20260709 13:12:28.326339 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:28.278662  1063 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:28.310747 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.299664 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:28.345184 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:28.345860 29157 raft_consensus.cc:2368] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14.
I20260709 13:12:28.349535 29152 raft_consensus.cc:1275] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:28.349714 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:28.350155 29154 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:28.224161   703 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.350731 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:28.351269 29151 raft_consensus.cc:2376] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11. Re-sending same reply.
I20260709 13:12:28.339495 29159 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:28.331359   852 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:28.340691   993 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:28.350054 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:28.294332  1103 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.283653  1104 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:28.336004 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 105510us
W20260709 13:12:28.354403 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 57749.1us
I20260709 13:12:28.355593 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:28.357067 29153 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:28.360477 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:28.216960  1075 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.283883  1102 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:28.364814  1110 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.365463  1110 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.367877  1110 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:28.371165 29122 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.299042   934 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.371299 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:28.298952  1113 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.325306  1121 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:12:28.405628 29010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.012s	sys 0.005s
I20260709 13:12:28.352632   780 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:28.359083   523 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:28.411919 28975 leader_election.cc:400] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14.
I20260709 13:12:28.343205   971 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:28.417495 29010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.013s	sys 0.005s
W20260709 13:12:28.404974  1096 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 13:12:28.333297  1085 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:12:28.425148 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:28.425345 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 54404.8us
I20260709 13:12:28.425709 29079 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:28.426039 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.420446 28975 leader_election.cc:403] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:12:28.420812 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:28.426507 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 144760us
I20260709 13:12:28.427299 29157 raft_consensus.cc:2376] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12. Re-sending same reply.
I20260709 13:12:28.428328 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:28.429060 29157 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:28.429185 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:28.429772 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:28.397161  1057 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:28.430480 29157 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:28.405567   931 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:28.429888 29009 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:12:28.408527   537 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:28.404858  1121 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:28.416730 32630 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:28.441077 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:28.452303 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:28.447242 29122 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.453163 29002 raft_consensus.cc:2410] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:28.352851  1069 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.358275  1056 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.398887  1110 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:12:28.445686 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:28.465090 29157 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:28.465997 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:28.409821  1102 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:28.424553  1075 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:28.420881   826 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:28.421496   780 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:28.416563  1132 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:28.456523 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:28.488601 29122 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.489923 29122 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.490864 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:28.462864  1094 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:12:28.463891   931 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:28.385931  1103 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.438747   683 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:28.445094   537 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:28.507607   537 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:28.469949  1110 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:28.508193  1110 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:28.509045  1110 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.510181  1110 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:28.511018 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:28.511607 29155 raft_consensus.cc:2376] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11. Re-sending same reply.
I20260709 13:12:28.506647  1075 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14.
I20260709 13:12:28.513901  1074 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:28.514357  1074 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:28.425401  1105 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:28.392822   934 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.353374  1064 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.440441   523 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:28.441963  1123 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.486802 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.454445  1121 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:28.459056   993 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.494536  1132 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:28.550366 29715 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:12:28.546591 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2365) took 1245 ms. Trace:
W20260709 13:12:28.531363 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44643.2us
I20260709 13:12:28.499884 29079 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:28.444398  1145 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:28.424715   777 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.408818  1113 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.439033  1134 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:12:28.506798   828 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:28.506649   971 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.440531   988 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.531858  1104 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:28.425541   852 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.493947   780 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:28.594084  1125 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:12:28.571810 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:28.599161 29151 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:12:28.595002 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:28.510522  1082 consensus_meta.cc:287] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.659s	user 0.007s	sys 0.001s
I20260709 13:12:28.572381  1150 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.531353  1063 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:28.525722   902 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:28.583487  1151 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:28.503602   703 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:28.592818   783 consensus_meta.cc:287] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.782s	user 0.009s	sys 0.002s
I20260709 13:12:28.566140  1132 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:28.532367  1123 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.588186  1145 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:28.612689   828 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:28.568207  1148 raft_consensus.cc:2764] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:12:28.527909  1137 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:28.570297   993 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:28.667402  1130 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:12:28.662011  1141 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:28.670408  1118 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:12:28.676338   912 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.000s
I20260709 13:12:28.672284  1082 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:28.585804   931 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.658754  1150 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.578843  1102 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.566673 29005 rpcz_store.cc:276] 0709 13:12:27.301159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.302240 (+  1081us) service_pool.cc:224] Handling call
0709 13:12:27.302558 (+   318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:27.317083 (+ 14525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:27.317116 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:27.317652 (+   536us) log.cc:844] Serialized 52 byte log entry
0709 13:12:27.690074 (+372422us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:27.690127 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:27.690220 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:28.534106 (+843886us) raft_consensus.cc:1685] finished
0709 13:12:28.534133 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:28.546442 (+ 12309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9418,"raft.queue_time_us":23022,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":11102,"thread_start_us":320080,"threads_started":1,"wal-append.queue_time_us":572354}
W20260709 13:12:28.633450 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 142629us
I20260709 13:12:28.645150 29154 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:28.653131 29158 raft_consensus.cc:2410] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:12:28.843920 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2537) took 1148 ms. Trace:
I20260709 13:12:28.888628 29159 rpcz_store.cc:276] 0709 13:12:27.695194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.850846 (+155652us) service_pool.cc:224] Handling call
0709 13:12:27.851114 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:28.351922 (+500808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:28.843113 (+491191us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54003f6bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:28.843645 (+   532us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1122585344}
I20260709 13:12:28.890829   902 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:28.593376   523 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.891680 32570 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.774801 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2367) took 1472 ms. Trace:
W20260709 13:12:28.893627 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.893183 32570 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.897249 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:28.897828   986 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.673s	user 0.016s	sys 0.000s
I20260709 13:12:28.898157   919 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:28.705601  1064 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:28.688107   783 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:28.727421  1061 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.001s
I20260709 13:12:28.706422  1119 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:12:28.691486  1070 consensus_meta.cc:287] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.680s	user 0.005s	sys 0.006s
I20260709 13:12:28.849627   828 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.775429 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2531) took 1462 ms. Trace:
I20260709 13:12:28.796587 29085 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [9.5-10.6]
I20260709 13:12:28.852221  1013 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:28.724802  1103 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:28.785329 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:28.853860   934 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:28.695212  1118 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:28.694762  1141 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:28.714680  1056 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:28.769209 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:28.792464 29077 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.008s	sys 0.012s
I20260709 13:12:28.795084 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:28.795008 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2134) took 1241 ms. Trace:
I20260709 13:12:28.787843 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.693195   993 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:28.858282  1059 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:28.732725  1121 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.717175   988 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:28.777272 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:28.801705 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:28.813954 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:28.717514  1069 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:28.692756  1092 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.780s	user 0.005s	sys 0.001s
I20260709 13:12:28.689965  1074 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.718636  1113 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:28.863395 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:28.935501 29077 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.009s	sys 0.012s
I20260709 13:12:28.863530  1138 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:28.691352  1135 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:12:28.864225  1105 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.693233  1145 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.705852  1150 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:28.716701   967 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:28.867143   777 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:28.737025  1132 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.718866  1063 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.869050 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:28.718043  1093 log.cc:927] Time spent T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.831s	user 0.000s	sys 0.004s
I20260709 13:12:28.872491   683 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:28.880637  1068 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.881106  1162 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.963048  1162 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.966111  1162 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:28.966243  1132 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:28.881925  1072 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.882828  1109 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:28.890749 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:28.711272  1100 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.696s	user 0.004s	sys 0.000s
I20260709 13:12:28.694564  1123 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:28.892490 29001 rpcz_store.cc:276] 0709 13:12:27.301934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.398275 (+ 96341us) service_pool.cc:224] Handling call
0709 13:12:27.419635 (+ 21360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:27.440593 (+ 20958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:27.440665 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:27.460628 (+ 19963us) log.cc:844] Serialized 26 byte log entry
0709 13:12:28.277891 (+817263us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:28.277945 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:28.278033 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:28.773905 (+495872us) raft_consensus.cc:1685] finished
0709 13:12:28.774002 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:28.774446 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19793,"raft.queue_time_us":21677,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6785,"thread_start_us":816904,"threads_started":1,"wal-append.queue_time_us":1219929}
I20260709 13:12:28.899369 32570 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:28.899735  1151 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:28.900879   932 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.695523   912 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:28.731467  1076 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.919s	user 0.010s	sys 0.000s
I20260709 13:12:28.885417  1155 consensus_queue.cc:1048] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:28.851792   982 consensus_meta.cc:287] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 1.330s	user 0.011s	sys 0.000s
I20260709 13:12:28.908627 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:28.902961   986 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.917418 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:28.910030   902 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:28.940760  1138 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:28.941422 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 156600us
I20260709 13:12:28.943815  1061 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.946307  1118 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.881963  1106 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:12:28.948194  1141 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:28.948662  1116 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 13:12:28.949201 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 161903us
I20260709 13:12:28.994797 29081 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:12:28.923045  1119 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:29.002851   683 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.004597  1076 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.968910  1068 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.004698 29156 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:29.021611  1074 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:28.968158 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 187248us
I20260709 13:12:28.910804 29010 rpcz_store.cc:276] 0709 13:12:27.312366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.735783 (+423417us) service_pool.cc:224] Handling call
0709 13:12:27.780594 (+ 44811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:27.814203 (+ 33609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:27.816493 (+  2290us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:27.816943 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:12:28.421755 (+604812us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:28.421808 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:28.421880 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:28.774774 (+352894us) raft_consensus.cc:1685] finished
0709 13:12:28.774821 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:28.775160 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13827,"raft.queue_time_us":68497,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":26905,"thread_start_us":603467,"threads_started":1,"wal-append.queue_time_us":834038}
I20260709 13:12:29.022706  1109 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:28.975593 29084 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:29.025017 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:29.025942 29084 raft_consensus.cc:2393] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:29.026173 28975 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:29.026793 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:29.028133 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2364) took 1729 ms. Trace:
I20260709 13:12:29.029222 29159 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:29.030887   934 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:29.031391   934 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.920392 29080 rpcz_store.cc:276] 0709 13:12:27.553265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.732669 (+179404us) service_pool.cc:224] Handling call
0709 13:12:27.732934 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:28.058599 (+325665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:28.058647 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:28.079673 (+ 21026us) log.cc:844] Serialized 26 byte log entry
0709 13:12:28.322909 (+243236us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:28.322962 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:28.323068 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:28.794474 (+471406us) raft_consensus.cc:1685] finished
0709 13:12:28.794515 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:28.794797 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55316,"raft.queue_time_us":152856,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":13826,"thread_start_us":394308,"threads_started":2,"wal-append.queue_time_us":552271}
I20260709 13:12:28.947973  1070 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.033061 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:29.034562  1070 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:28.959578 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:29.035760 29085 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:28.992128 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 48573.1us
I20260709 13:12:28.971745  1013 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.037881  1138 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.038940  1013 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.039510 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:28.996304  1064 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:28.994323  1105 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.055104  1105 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:28.999006   986 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.000437 29154 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 15.
I20260709 13:12:29.001808  1113 consensus_queue.cc:1048] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:29.002632 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55310.2us
I20260709 13:12:28.996094 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:29.039736   932 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.031780   934 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:28.992936   982 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.074059   881 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:28.994488 29082 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:29.075555  1121 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.005223  1072 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:28.984856   783 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.043085  1109 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.043968  1119 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:29.086834 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.046106   988 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:29.088279   988 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:29.009549   537 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.013612 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2366) took 1712 ms. Trace:
I20260709 13:12:29.018296   912 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:29.016330 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52258.2us
I20260709 13:12:29.016660  1126 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:29.051399  1068 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.027885 29084 raft_consensus.cc:2410] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:12:29.020224 29157 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:28.987946  1059 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.018594 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:29.073199 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 77394.2us
I20260709 13:12:29.073918   932 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.042583   967 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:29.016768   530 consensus_meta.cc:287] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.757s	user 0.006s	sys 0.007s
I20260709 13:12:29.101998   530 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.017413  1069 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:29.045295 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 154897us
I20260709 13:12:29.007241  1076 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.092435 29004 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:29.014179  1116 consensus_queue.cc:1243] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:12:29.070791  1155 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:29.028688 29008 rpcz_store.cc:276] 0709 13:12:27.298985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.299107 (+   122us) service_pool.cc:224] Handling call
0709 13:12:27.299390 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:27.299722 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:27.299754 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:27.300222 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:12:27.434386 (+134164us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:27.434528 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:27.434733 (+   205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:28.717429 (+1282696us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:29.027514 (+310085us) raft_consensus.cc:1685] finished
0709 13:12:29.027564 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:29.027881 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1024,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":329774848,"thread_start_us":133530,"threads_started":1,"wal-append.queue_time_us":464422}
I20260709 13:12:29.099264  1119 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:29.100980  1072 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.083142   783 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:29.114190 29159 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:29.116266 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:29.117372 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:29.118263  1059 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.122745 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 779) took 1129 ms. Trace:
I20260709 13:12:29.127081 29001 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:29.126925 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.094651  1126 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:29.127847   780 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.129249 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.130908 29006 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:29.131263 29010 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:29.127816 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.093156 29007 rpcz_store.cc:276] 0709 13:12:27.301429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.301536 (+   107us) service_pool.cc:224] Handling call
0709 13:12:27.301839 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:27.302156 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:27.302185 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:27.302591 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 13:12:27.513463 (+210872us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b48000801d8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411a76aaa1 00007f411a761b37 00007f411a7619ca 00007f411a85926f 00007f411a858f37 00007f411a86d55c 00007f411a86d508 00007f411a86d488 00007f411a86d443 00007f411a86c6ec 00007f4119796a96
0709 13:12:27.787944 (+274481us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:27.787985 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:27.788095 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:28.710554 (+922459us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:29.013085 (+302531us) raft_consensus.cc:1685] finished
0709 13:12:29.013125 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:29.013401 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14977,"raft.run_cpu_time_us":2447,"raft.run_wall_time_us":186046,"spinlock_wait_cycles":36464896,"thread_start_us":485096,"threads_started":1,"wal-append.queue_time_us":712297}
W20260709 13:12:29.135874 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 155626us
I20260709 13:12:29.140172 29154 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:29.141593   537 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:29.143996 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:29.149302  1123 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.196879  1123 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:29.198704  1163 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.461s	user 0.005s	sys 0.000s
W20260709 13:12:29.200201 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 107099us
W20260709 13:12:29.204424 28895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 60721.3us
I20260709 13:12:29.204972  1159 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:29.205832  1061 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.205650  1159 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.209448 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.211226 29005 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:29.151700 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:29.152508  1151 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.154264   932 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:29.155288 32570 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:29.156222   986 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.157011 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2541) took 1460 ms. Trace:
W20260709 13:12:29.157117 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 138728us
I20260709 13:12:29.158339  1102 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:29.226639 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:29.167841  1165 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:12:29.087443   982 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.177330  1126 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:29.177742 29708 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:12:29.178179  1172 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:12:29.179116 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2374) took 1737 ms. Trace:
I20260709 13:12:29.180773 29153 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:29.181226 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.175325 29077 rpcz_store.cc:276] 0709 13:12:27.992972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.994772 (+  1800us) service_pool.cc:224] Handling call
0709 13:12:28.087083 (+ 92311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:28.087508 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:28.087547 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:28.088094 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:12:29.013832 (+925738us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:29.013916 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:29.014038 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:29.014097 (+    59us) raft_consensus.cc:1685] finished
0709 13:12:29.014115 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:29.122414 (+108299us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b54001ad1d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:29.122673 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1077,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":21242,"spinlock_wait_cycles":211555200,"thread_start_us":847619,"threads_started":1,"wal-append.queue_time_us":645186}
I20260709 13:12:29.186710   828 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:29.173516 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:29.187395 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:29.191753 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.238670 29004 rpcz_store.cc:276] 0709 13:12:27.441914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.466137 (+ 24223us) service_pool.cc:224] Handling call
0709 13:12:29.178878 (+1712741us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400184cd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:29.179040 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3676091136}
I20260709 13:12:29.211386  1155 raft_consensus.cc:697] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:29.199716  1123 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:29.219601 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 112003us
I20260709 13:12:29.248502 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:29.163066  1170 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.074s	user 0.000s	sys 0.002s
I20260709 13:12:29.226328 29158 rpcz_store.cc:276] 0709 13:12:27.696299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.790066 (+ 93767us) service_pool.cc:224] Handling call
0709 13:12:29.156426 (+1366360us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b54003f70d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:29.156765 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1982021376}
I20260709 13:12:29.229557 29079 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:29.233279 29001 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:29.237906 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2529) took 2038 ms. Trace:
I20260709 13:12:29.244407 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.253990   986 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.238781 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.256866 29006 raft_consensus.cc:2410] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:12:29.257723 29007 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:29.258040   986 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.259389 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:29.252944 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:29.255764 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:29.256798 29001 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:29.211973  1159 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.171986  1069 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.262184 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.255718 29156 rpcz_store.cc:276] 0709 13:12:27.199796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.199955 (+   159us) service_pool.cc:224] Handling call
0709 13:12:29.213396 (+2013441us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400054e54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:29.237836 (+ 24440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3767684992}
I20260709 13:12:29.229353  1059 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.266885 29003 raft_consensus.cc:2410] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:12:29.269125 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.277594 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:29.281956 29709 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:29.289669 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.169782   988 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:29.293830 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 142296us
I20260709 13:12:29.297590 29713 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:12:29.297813 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:29.298568   932 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:29.298972   932 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:29.299052 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:29.300132 29709 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:29.299991   932 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.300441 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:29.300153 29083 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:29.301486   932 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:29.303164 29008 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:29.304574  1123 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:29.305576   996 consensus_queue.cc:1175] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "06ae4197fd784525b695a9b16c8094ea" responder_term: 11 status { last_received { term: 11 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 11 index: 8 } }
I20260709 13:12:29.307328  1059 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.309357   669 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:29.309675  1059 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.309998   993 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:29.309705   537 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:29.310947   683 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:29.311344   683 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:29.312512 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:29.315924 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 219449us
I20260709 13:12:29.316576   780 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.317026 29048 leader_election.cc:400] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
I20260709 13:12:29.318462 29048 leader_election.cc:400] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:29.274098   912 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:29.320858   993 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:29.321329   993 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.321734   993 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:29.322525  1069 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.269448   523 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:12:29.312428   683 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.324218   523 raft_consensus.cc:3055] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:29.324612   683 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:29.324702   523 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:29.326071 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56123.8us
I20260709 13:12:29.325877   523 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.327165   523 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:29.328090   530 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.328418 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:29.328856   530 consensus_queue.cc:1059] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:29.329826   530 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:29.330189   530 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:29.330566   530 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:29.332141  1102 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:29.332352 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.332685  1102 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:29.333304 29007 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:29.334334 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.333997  1102 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.335116 29007 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:29.291072 29682 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:29.337550 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 72382.9us
I20260709 13:12:29.338068 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.338493 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:29.339291 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.339505  1132 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:29.340087  1132 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:29.340420 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.341270 29154 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:29.342507 29152 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:12:29.343152 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.319953 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:29.345692   989 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:29.312974 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:29.315135 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:29.303956 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:29.347992 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.340576 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.350070   982 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.350378 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:29.350718   982 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.351114 29077 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:29.282164  1151 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.341284  1132 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.356491 29010 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:29.353684 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:29.360512 29076 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:12:29.365743 29076 pending_rounds.cc:85] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 8
I20260709 13:12:29.355777 29157 raft_consensus.cc:2410] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:29.355041 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:29.335789  1102 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:29.367388 29008 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:29.366007 29076 pending_rounds.cc:107] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:12:29.383602 29076 raft_consensus.cc:2889] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:29.345511  1141 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:29.386852 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 71794us
I20260709 13:12:29.387205 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:29.391708 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:29.404302 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 86030us
I20260709 13:12:29.405506 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:29.408259  1124 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:29.408387 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.408854  1124 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.409281  1124 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:29.412364 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58807.7us
I20260709 13:12:29.412743 29077 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:12:29.405690 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 50809.5us
I20260709 13:12:29.352996   982 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:29.405520 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.420775 29079 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:29.431892 29083 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 7. Current term is 8. Ops: [5.8-7.9]
I20260709 13:12:29.390374  1113 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:29.434950 29082 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:29.432068 29008 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:29.393980  1076 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.361581   902 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.437772 29159 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:29.440079 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.423949  1119 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:29.441749 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:29.451200 29085 raft_consensus.cc:2410] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:12:29.449960 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.463709 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:29.464212 29156 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:29.467692 29154 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:29.467604 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:29.467571 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:29.476089 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:29.465939 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.477739 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:29.480265 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:29.476770 29713 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:29.483659 29153 raft_consensus.cc:2393] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:29.353160  1155 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.486493 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.488521 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.442380   934 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.487619 29083 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:29.500178 32630 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:29.501061 32630 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:29.504770 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.505100 29158 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:29.506209   780 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:29.507154 29709 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:29.465183  1076 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.520473 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.521379 29152 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:29.526186 29002 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:29.502511 32630 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.517848  1118 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:29.518715   780 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:29.524971   986 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:29.517611  1119 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:12:29.554894 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.545732   780 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:29.545897   912 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:29.566044 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.550092   982 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:29.571488 29010 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:29.567117  1015 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.564114  1119 consensus_queue.cc:1243] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:29.568753  1074 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.589649 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:29.592698 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:29.593549 29002 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:29.596204 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.574370  1141 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:29.611152 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 144621us
I20260709 13:12:29.614331 29008 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:29.622622 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.623287 29158 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:29.625066 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:29.619827  1076 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.602752   902 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:29.628283  1119 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:29.586771   982 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.722211   971 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.591495   986 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.734364 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.588524  1015 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.603621   912 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.607544  1074 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:29.740432 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 189071us
I20260709 13:12:29.742926 29158 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:29.722620  1063 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.756024 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.756697 29153 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:29.760860 29009 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:12:29.747538   934 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.761235 29009 pending_rounds.cc:85] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 5
I20260709 13:12:29.766331 29009 pending_rounds.cc:107] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:12:29.766597 29009 raft_consensus.cc:2889] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:29.768966  1127 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:12:29.769270  1157 log.cc:927] Time spent T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
I20260709 13:12:29.777340  1015 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.780820  1069 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.781281   912 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.781415  1069 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.783561  1074 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:29.787318 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 222743us
I20260709 13:12:29.783809  1069 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:29.758976  1119 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.827836 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.727530   780 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:29.831243 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 64070.5us
I20260709 13:12:29.834905 29002 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:29.762131   993 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:29.845036 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.851845 29708 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:29.816483   971 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.860389 29158 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:12:29.854378   683 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.004s
I20260709 13:12:29.816725  1063 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.847012  1069 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:29.875214 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:29.872960 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:29.879698 29008 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:29.884739 29152 raft_consensus.cc:2410] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:12:29.873756 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:29.889814 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:29.892314   934 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:29.893791   780 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:29.892608  1124 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.896411 29153 raft_consensus.cc:2393] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:29.898025  1124 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:29.900521  1126 consensus_meta.cc:287] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.003s
I20260709 13:12:29.901077  1126 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.902073  1113 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:29.904865 29717 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:12:29.904866  1188 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:12:29.901504  1151 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:29.918550 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 55541us
I20260709 13:12:29.862613   912 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:29.926939   902 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:29.927176  1069 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.927922 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.932662   523 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.861353  1119 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:29.937376 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:29.936710   902 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.939795 29158 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:29.940495 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 77782us
W20260709 13:12:29.937724   783 consensus_meta.cc:287] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.759s	user 0.012s	sys 0.004s
I20260709 13:12:29.948709 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:29.950729 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:29.957983   982 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.960237   986 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.961514 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:29.962641 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:29.966077  1208 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:29.966989  1102 consensus_meta.cc:287] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.004s
I20260709 13:12:29.947037   902 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:29.948307  1070 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.953481   912 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.977543 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:29.977846   986 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:29.978468   993 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:29.978855  1063 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.982085 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:29.983546 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:29.985881 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.986042 29004 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:12:29.967567  1061 raft_consensus.cc:2764] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:12:29.990036 29050 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:30.003458 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:30.004123 29001 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:29.981012  1155 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:29.994783 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.996423 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:29.984234  1059 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:29.952335   783 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:29.968427  1069 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:29.941594   523 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.002682 29007 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:30.005198 28975 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:30.006814 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60651.2us
I20260709 13:12:30.012079 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:29.988721  1076 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:30.024840 29004 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:30.026372 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:30.027671 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 59998.9us
I20260709 13:12:30.028381 29158 raft_consensus.cc:2368] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
W20260709 13:12:30.028839 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 67421.6us
I20260709 13:12:30.033219 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:30.037603 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.037075 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.039552 29001 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:30.039659 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.013264  1061 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:30.041936  1059 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.043463   986 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:30.043843  1192 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:12:30.044190   530 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.004s
I20260709 13:12:30.044672 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.044560   996 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:30.045275 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.045192  1151 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:30.051862   523 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.053218  1200 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:30.088984 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:30.027711   971 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.043500 29125 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:29.989239  1070 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.062757 29007 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
W20260709 13:12:30.093380 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 103367us
I20260709 13:12:30.078794 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.011804  1127 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:12:30.054896   993 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:30.086511 28975 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.089293 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.089623 29077 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:30.061877   982 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:30.078215   683 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:30.098685 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.098682 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:30.094404   828 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:12:30.108088  1070 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.108340 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1672) took 1324 ms. Trace:
I20260709 13:12:30.103912 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:30.108700 29076 rpcz_store.cc:276] 0709 13:12:28.783614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.787676 (+  4062us) service_pool.cc:224] Handling call
0709 13:12:28.788459 (+   783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:29.392771 (+604312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:30.108114 (+715343us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54000460d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:30.108285 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314268544}
I20260709 13:12:30.103572   780 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:12:30.110216 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 66796.1us
I20260709 13:12:30.111316 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:30.112444  1061 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:30.113128  1141 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:30.113233  1061 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:30.113718  1141 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.114581 29122 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.114843 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.115689 29151 raft_consensus.cc:2393] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:30.116276  1123 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:30.116477 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.117012  1123 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.114724  1141 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.118129 29151 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:12:30.118230  1123 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.120175  1193 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:12:30.098487 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:30.133435 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 47040.1us
I20260709 13:12:30.103379 29155 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: [10.12-10.12]
I20260709 13:12:30.102411 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:30.134128 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.115811 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.136904 29081 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:30.138543 29076 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:30.067776 32630 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:12:30.139372 28975 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.084681   530 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.163872 29008 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:30.164170 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.165241   530 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.165396 29076 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:30.173337 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:30.174130 29085 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:30.174799 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:30.175998 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:30.173394 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:30.179282  1059 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:30.181129   912 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:30.140213  1200 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:30.182729  1200 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.183163  1200 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:30.131228   683 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.185250   683 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:30.101477   982 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.188524   982 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:30.190831 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51563.7us
I20260709 13:12:30.142516 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.144315 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:30.144976  1212 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:30.137754  1124 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:12:30.152567   902 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.156133 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.157474 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.158710 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:30.159157 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:30.160629 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.161441   783 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:30.101431   993 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:30.162261   934 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:30.163172   971 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:30.196238 29007 raft_consensus.cc:2368] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
W20260709 13:12:30.199932 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 55690.9us
I20260709 13:12:30.200172 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:30.201611 29077 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
W20260709 13:12:30.202541 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 86826.9us
I20260709 13:12:30.207238 29079 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:30.206061 29084 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:30.205267 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:30.209441 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.209441 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:30.210011 29082 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:30.209962   993 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:30.229117   971 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.233023   971 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:30.210641 29081 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:30.204406  1076 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:12:30.211052 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:30.203845   828 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:12:30.211434 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.212667 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:30.239609 29156 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:12:30.213061  1209 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:12:30.200114   967 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:30.218621  1126 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.218513   982 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.222536   909 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:12:30.244336 29158 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:30.224834   669 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:12:30.226110  1181 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.702s	user 0.000s	sys 0.002s
I20260709 13:12:30.227528 32570 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:30.231930   988 raft_consensus.cc:3055] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:30.203686   996 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:30.237898 29050 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:30.242151 29002 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:30.204669  1070 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:30.247218 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.227638   934 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:30.253576 32570 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.259229 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:30.241729 32630 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:12:30.263911 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.266017 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:30.249861   669 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.273109 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.273842 29007 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:30.252698   988 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.240428  1015 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:12:30.257949  1212 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:30.259627   934 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:30.257021   967 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:30.278044 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 48084.3us
I20260709 13:12:30.275091 29159 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:30.279834   988 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:30.280723  1190 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.305s	user 0.000s	sys 0.004s
I20260709 13:12:30.280987  1151 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:30.281559   669 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.283914 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 670) took 1380 ms. Trace:
I20260709 13:12:30.284431 29085 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:30.288033 32570 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.289681   902 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:30.307301 29154 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:30.307636   523 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:30.308061  1121 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.308506  1126 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:30.308647  1121 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.309089  1126 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.310941  1121 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.311334  1126 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.312143   523 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:30.313104   523 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.314059   523 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:30.315411 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 787) took 1045 ms. Trace:
I20260709 13:12:30.316758 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.317366 29001 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:30.317984 29154 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:12:30.318637 29001 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:30.318835   852 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:30.291605  1119 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.248313   537 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:12:30.294163 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:30.325745 29006 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 9: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:30.258580  1070 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:30.302608   967 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:30.296381 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:30.294174 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:30.377995 29008 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:30.298485 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:30.300957 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:30.376945 28895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 80608.4us
I20260709 13:12:30.254801   783 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:12:30.305186 29155 rpcz_store.cc:276] 0709 13:12:28.903355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.916403 (+ 13048us) service_pool.cc:224] Handling call
0709 13:12:28.948800 (+ 32397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:30.137111 (+1188311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:30.283651 (+146540us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400056254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:30.283836 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2512769408}
I20260709 13:12:30.315775 29010 rpcz_store.cc:276] 0709 13:12:29.270243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:29.587195 (+316952us) service_pool.cc:224] Handling call
0709 13:12:29.598896 (+ 11701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:29.921001 (+322105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:29.921045 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:29.940374 (+ 19329us) log.cc:844] Serialized 26 byte log entry
0709 13:12:30.190897 (+250523us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:30.190938 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:30.315019 (+124081us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54003e21d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:30.315071 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:30.315117 (+    46us) raft_consensus.cc:1685] finished
0709 13:12:30.315132 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:30.315354 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":569,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":10472,"spinlock_wait_cycles":726598400,"thread_start_us":185995,"threads_started":1,"wal-append.queue_time_us":292221}
I20260709 13:12:30.357589 29005 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:30.356815 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.358564 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.388535 29081 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:30.359440 29153 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:30.395388 28975 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:30.361366  1070 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:30.362772 29006 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:30.365168   993 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:30.365999 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45293.3us
I20260709 13:12:30.356942  1127 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:30.376010 29001 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:12:30.302120   988 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:30.381466 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.382849 29003 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:30.419556 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.408110 29001 pending_rounds.cc:85] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 8
I20260709 13:12:30.420590 29001 pending_rounds.cc:107] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:12:30.420907 29001 raft_consensus.cc:2889] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:30.423300 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 124867us
I20260709 13:12:30.424283 29050 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:30.429960 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.430783 29010 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:30.304749  1117 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.427064 29122 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.448866   993 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:30.417200  1116 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:30.407944  1127 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:30.451593  1127 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.452387 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.453197  1127 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:30.436187   828 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:12:30.456707   783 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:30.457361   783 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.458498   783 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.460773 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.447803 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:30.462534 29156 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:30.463649 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.492385 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:30.516216 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 92033.2us
I20260709 13:12:30.538390 29083 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:30.485483  1126 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.524715 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:30.539944 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 280719us
I20260709 13:12:30.491861  1212 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:30.491501   993 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:30.442587  1117 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.491706  1119 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.574095 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.522718   912 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:30.523646  1015 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:12:30.596316 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:30.575582 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 155087us
I20260709 13:12:30.586959 29156 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:30.568945   780 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.537611  1151 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:12:30.569304  1214 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.624573 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:30.626332 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:30.627187   828 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:30.627907   828 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.630508   828 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.556213  1208 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:12:30.571192  1141 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:12:30.648960 29010 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:30.572379  1126 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:30.681257 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 56793us
I20260709 13:12:30.680791 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.685053 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:30.647262   780 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:30.686911 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 115008us
I20260709 13:12:30.688987 29050 leader_election.cc:400] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:30.690307 32570 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:30.691429 29050 leader_election.cc:403] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:12:30.616683   993 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.703707   897 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:30.704752   993 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.710819 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.712106 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:30.713014  1230 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:30.713411  1230 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:30.714350  1230 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.715514  1230 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:30.715857 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.716609 29009 raft_consensus.cc:2393] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 15: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:30.717337 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.717859 29009 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:30.718677 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.719403   780 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:30.719929 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 97625.7us
I20260709 13:12:30.720700 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.722927 29007 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:30.719846 29009 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:30.695483 29682 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:30.726477 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.727181 29004 raft_consensus.cc:2393] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 14: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:30.699795   934 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.731009   934 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.733120 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.734154 29083 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:30.734781 29081 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:12:30.735119 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.735962 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:30.737090 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.737442 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.737671 29081 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:30.737990 29083 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:30.738101 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.572510  1200 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.740738 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:30.741561 29715 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:30.742267  1200 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.743537 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:30.745987 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.748373 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:30.749979   902 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:30.750533   902 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.754541  1061 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:30.754961  1061 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:30.762562 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:30.763425 29079 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:30.600246  1212 consensus_queue.cc:1059] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:12:30.698956  1151 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.613909  1145 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.267s
I20260709 13:12:30.856000 29007 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:30.751461   902 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.857630   932 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:30.858165   902 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:30.858422 29007 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:30.858758   996 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.859213   996 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.701786 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.702800  1126 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.609750  1119 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:30.724480   683 consensus_meta.cc:287] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.001s
I20260709 13:12:30.696488  1221 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.839725 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:30.839996 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:30.840026 29005 raft_consensus.cc:2368] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14.
I20260709 13:12:30.776515 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:30.737221  1121 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:30.810598 29048 leader_election.cc:400] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
I20260709 13:12:30.771657 32570 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:30.891140 29157 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:30.690595  1214 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.695076   912 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.900198  1063 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.900735  1063 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.901731   912 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:30.854725   971 consensus_meta.cc:287] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.002s
I20260709 13:12:30.902420   971 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.867244 29003 raft_consensus.cc:2393] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:30.903883 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:30.904695   971 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:30.905449 29003 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:30.905699 29002 raft_consensus.cc:2368] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
W20260709 13:12:30.868734 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 183755us
I20260709 13:12:30.906352 32570 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.908725 32570 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:30.613937   909 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:12:30.855166  1151 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:30.874817 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 185985us
W20260709 13:12:30.866974   967 consensus_meta.cc:287] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.532s	user 0.004s	sys 0.006s
W20260709 13:12:30.877931 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 157173us
I20260709 13:12:30.879496  1212 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:30.652256  1069 consensus_queue.cc:1059] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.233s
I20260709 13:12:30.858174   932 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:30.855842  1102 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:12:30.755810  1061 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.884030  1117 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:30.890827  1192 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:30.897945 29708 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:30.898164 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:30.899887  1214 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:30.902774  1063 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:30.964640 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 364191us
I20260709 13:12:30.877828  1238 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:30.873967 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:30.996177 29079 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:30.890595  1123 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:30.888149 29004 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:30.914726 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 138263us
I20260709 13:12:30.962466 29154 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:30.920218 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 109832us
I20260709 13:12:30.931782  1212 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:30.945804 29050 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:30.909898 29083 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:30.894321  1121 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:30.893172   537 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:30.967608 29717 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:30.972513 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:30.977483 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:30.977907   669 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:30.978859  1208 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.979676 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:31.008658 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:31.029834 29082 raft_consensus.cc:2393] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:31.029987 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52618us
W20260709 13:12:31.000893 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.101s (SENT) [suppressed 16 similar messages]
I20260709 13:12:30.980525  1116 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:30.991159 29005 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:31.002796 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60501.7us
I20260709 13:12:30.999804 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:31.003391 29159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.020s	sys 0.010s
I20260709 13:12:30.965420 29683 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:31.029325 29084 raft_consensus.cc:2393] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:30.895064   683 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:30.986459   982 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:31.051976 29151 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:31.053043 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:31.053869 29004 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
W20260709 13:12:31.043501 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 97729.9us
W20260709 13:12:31.052322 29159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.020s	sys 0.010s
I20260709 13:12:31.055234 29122 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:31.056880 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 56030us
I20260709 13:12:31.059051 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:31.060397 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:31.060441 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 95105.4us
I20260709 13:12:31.061408 29713 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:30.932622   967 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.063688 29683 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:31.066118 29048 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.067014 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:30.943739   988 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:12:31.012295  1145 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:31.011261  1121 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:31.010360  1240 raft_consensus.cc:3055] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:30.937055  1232 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.134s
I20260709 13:12:31.055920   934 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:31.051249  1061 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:31.076407 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:31.081892   932 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:31.081707  1059 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:31.071491   897 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:31.082826   828 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:31.113837   828 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:31.114379   828 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:31.073544  1015 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:31.116303  1015 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.116809  1015 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:31.116945  1145 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:31.117981  1145 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.064415   971 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:12:31.072026  1243 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:12:31.121711  1243 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:31.122184  1243 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:31.056129   912 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:31.064253  1200 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:12:31.084808  1070 consensus_meta.cc:287] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.643s	user 0.006s	sys 0.003s
I20260709 13:12:31.124588  1070 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.028999   783 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:12:31.126562  1070 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:31.126593   996 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:30.965147  1192 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.127736 29085 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:31.097447  1106 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:31.128815  1106 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.129189  1106 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:31.132831  1119 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:31.133435  1192 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:31.133471 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.086231  1151 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:31.086738  1222 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:31.136371  1222 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.136739  1222 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:31.139024 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.123250  1243 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.140731  1243 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:31.141883  1212 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.143170  1212 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:30.909925  1221 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.145537 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:31.091878 29154 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:31.107597 29083 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:31.110169 29713 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:31.146451 29085 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:31.049731   537 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.111253 29078 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:31.112257  1159 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:31.113225   993 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:31.115866  1235 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:12:31.056962  1208 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.089818  1240 raft_consensus.cc:740] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:31.160693 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:31.153083  1113 consensus_queue.cc:1059] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:12:31.169015 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:31.169736 29714 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:31.166009  1151 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:31.163323   912 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:31.162948 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 96011.2us
I20260709 13:12:31.166842 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:31.184600 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 457059us
I20260709 13:12:31.205930 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:31.152455  1192 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:12:31.209198 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:31.209817 29007 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:31.215992 29076 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:31.216807 29083 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:31.217017 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:31.217780 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:31.219038 29714 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:12:31.219384 29714 pending_rounds.cc:85] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Aborting all ops after (but not including) 8
I20260709 13:12:31.219666 29714 pending_rounds.cc:107] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:12:31.220155 29714 raft_consensus.cc:2889] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:31.221733 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:31.220793 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:31.267935 29007 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:31.268298 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:31.176278  1151 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:31.267554 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:31.268867 29154 raft_consensus.cc:2376] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11. Re-sending same reply.
I20260709 13:12:31.269302 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:31.274513 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:31.283947 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:31.288321 29151 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:31.268342 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:31.283531 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 76719.8us
I20260709 13:12:31.296746 29155 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 7.
I20260709 13:12:31.164371   993 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.299814 29082 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:31.296419 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:31.303248 29083 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:31.365479 29078 raft_consensus.cc:2410] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:12:31.370752 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:31.447434 29154 raft_consensus.cc:2368] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
I20260709 13:12:31.371862 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:31.177778   988 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.126s
W20260709 13:12:31.394246 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 96887.4us
W20260709 13:12:31.368999  1209 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:12:31.425959 29005 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:31.434810 29152 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
W20260709 13:12:31.380668  1210 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:12:31.299201  1151 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:31.212673  1126 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.268860  1119 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.465548 29709 raft_consensus.cc:1240] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:31.218293  1070 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.177s
I20260709 13:12:31.467201 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:31.534374 29085 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:31.477483  1232 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 13:12:31.511611 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 441706us
I20260709 13:12:31.435345  1113 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:12:31.491703 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:31.461494   932 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.505422 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:31.491667 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:31.503192  1123 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:31.366918  1193 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:31.527097 29078 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:31.540856 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.542285 29156 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:31.545450 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:31.548014 29085 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:31.537879  1124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.016s	sys 0.098s
I20260709 13:12:31.433561  1240 consensus_queue.cc:260] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.501602  1222 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:31.551460 29001 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.007s	sys 0.006s
W20260709 13:12:31.565088 29001 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.007s	sys 0.006s
I20260709 13:12:31.560072 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:31.561859 29010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.011s	sys 0.002s
W20260709 13:12:31.571564 29010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.011s	sys 0.002s
W20260709 13:12:31.562839 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 503798us
I20260709 13:12:31.572762 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:31.582237 29085 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
W20260709 13:12:31.556380  1124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.016s	sys 0.098s
I20260709 13:12:31.536803  1119 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.607046 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.568117  1069 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:12:31.605731 28975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.029s	sys 0.022s
W20260709 13:12:31.634958 28975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.029s	sys 0.023s
W20260709 13:12:31.635346 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 635533us
W20260709 13:12:31.635385 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 421276us
I20260709 13:12:31.548619  1126 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.548997   932 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.573493   982 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:31.651226   986 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:31.651542 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 44584.6us
I20260709 13:12:31.655649 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.549840  1123 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.632768  1221 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:31.661080 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:31.661832 29080 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:31.662632 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:31.663189 29080 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:31.663345  1233 consensus_queue.cc:1059] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:12:31.665103  1257 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:12:31.665634  1106 consensus_meta.cc:287] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.000s
I20260709 13:12:31.666044  1106 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.667115 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:31.668922  1246 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:12:31.669210  1233 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:31.672427 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 681) took 1369 ms. Trace:
I20260709 13:12:31.645452  1208 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:31.651957   986 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:31.674469   986 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.672636 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:31.679600  1015 consensus_meta.cc:287] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.562s	user 0.005s	sys 0.005s
I20260709 13:12:31.684262  1232 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:31.688936 29159 rpcz_store.cc:276] 0709 13:12:30.303025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.303188 (+   163us) service_pool.cc:224] Handling call
0709 13:12:30.316615 (+ 13427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:30.357660 (+ 41045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:30.357688 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:30.358240 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.057537 (+699297us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:31.057577 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.095347 (+ 37770us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b54000947d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:31.095395 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:31.672133 (+576738us) raft_consensus.cc:1685] finished
0709 13:12:31.672159 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:31.672364 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1185,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":51550080,"thread_start_us":698804,"threads_started":1,"wal-append.queue_time_us":851307}
I20260709 13:12:31.640504  1200 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:31.701032 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:31.701826 29079 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.000s
I20260709 13:12:31.551949  1193 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.712136  1193 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:31.712427  1263 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:31.659469  1247 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 13:12:31.684237  1015 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.714067 29079 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:31.713932 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2421) took 1014 ms. Trace:
I20260709 13:12:31.731494 29082 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
W20260709 13:12:31.714067  1238 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.702s	user 0.003s	sys 0.007s
I20260709 13:12:31.733777 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:31.731251 29010 rpcz_store.cc:276] 0709 13:12:30.699563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.860299 (+160736us) service_pool.cc:224] Handling call
0709 13:12:30.860665 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:30.911838 (+ 51173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:30.911885 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:30.935539 (+ 23654us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.572093 (+636554us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:31.572150 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.572220 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:31.713732 (+141512us) raft_consensus.cc:1685] finished
0709 13:12:31.713754 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:31.713881 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4507,"raft.queue_time_us":12647,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":7369,"thread_start_us":567979,"threads_started":1,"wal-append.queue_time_us":748784}
I20260709 13:12:31.748813 29001 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:31.739573 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 217574us
I20260709 13:12:31.746485 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:31.752135 29080 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:31.755594 29078 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:12:31.752130 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:31.759208 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 103414us
I20260709 13:12:31.761361 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:31.723969  1250 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.713012  1263 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.768077 29122 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:31.771466 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 211487us
I20260709 13:12:31.712420  1233 consensus_queue.cc:1059] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:12:31.805313 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.805477 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:31.732929  1127 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.809861 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.815092 29048 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:31.752590  1200 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:31.753963  1238 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
W20260709 13:12:31.811584 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 43726.1us
I20260709 13:12:31.829763 29079 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:31.821184   932 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:31.822142   828 consensus_meta.cc:287] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.707s	user 0.007s	sys 0.003s
I20260709 13:12:31.781904  1106 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:31.782070  1193 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:31.766410  1069 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:31.834867 29079 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:12:31.833564   902 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:12:31.840806   897 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.841385  1263 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:31.843006  1015 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:31.815490  1121 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.000s
W20260709 13:12:31.843434  1249 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
W20260709 13:12:31.844259  1251 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:12:31.829210  1127 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:31.845391  1254 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:12:31.849855  1270 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:12:31.858783   537 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:31.861908   683 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:31.864711  1238 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:31.864848 29081 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:31.865358   993 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:31.810380  1250 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:31.815667   669 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.002s
W20260709 13:12:31.874212 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44861.1us
I20260709 13:12:31.808540  1126 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:31.825664 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:31.957437 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 49190.9us
W20260709 13:12:31.828567  1248 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.291s	user 0.001s	sys 0.001s
I20260709 13:12:31.816649  1119 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:31.881742 29001 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:31.893949 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:31.879928   897 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:31.963857 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 59605.6us
I20260709 13:12:31.874897   828 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.879563  1221 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 13:12:31.977829 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 212466us
I20260709 13:12:31.978257 32570 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.884855   988 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:12:31.982379 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.979908 32570 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:31.884442  1015 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:12:31.992218 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.987183 29083 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:31.994451 29083 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:31.995858 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:31.887974  1121 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:31.958592  1117 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:31.995438 29083 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.001129   669 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.012770   669 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.015293   669 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:32.015031 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 800) took 1022 ms. Trace:
I20260709 13:12:31.978730  1119 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.016891 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:31.995136  1270 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:12:32.017611 29083 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:32.021997 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:32.019913 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:31.991060   932 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:31.992339  1126 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:32.030580 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:32.031204 29002 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.033021 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:32.034324 29715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 565) took 1263 ms. Trace:
I20260709 13:12:32.010648  1193 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:32.028460  1263 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:31.994093  1151 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.018273  1200 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:32.018446 29003 rpcz_store.cc:276] 0709 13:12:30.992096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.992596 (+   500us) service_pool.cc:224] Handling call
0709 13:12:31.011636 (+ 19040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.011832 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:12:31.542378 (+530546us) log.cc:844] Serialized 14 byte log entry
0709 13:12:32.013259 (+470881us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:32.013571 (+   312us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:12:32.013944 (+   373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.014219 (+   275us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.014771 (+   552us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":535064,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":113869,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4448,"spinlock_wait_cycles":22590080,"thread_start_us":949404,"threads_started":3,"wal-append.queue_time_us":341961}
I20260709 13:12:32.034822 29715 rpcz_store.cc:276] 0709 13:12:30.765130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.765312 (+   182us) service_pool.cc:224] Handling call
0709 13:12:30.765732 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:30.766220 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:30.766260 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:30.766843 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.170371 (+403528us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:31.171262 (+   891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.171325 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.027963 (+856638us) raft_consensus.cc:1685] finished
0709 13:12:32.027990 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.028178 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1321,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":62,"thread_start_us":400344,"threads_started":1,"wal-append.queue_time_us":771495}
I20260709 13:12:32.025218  1279 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:32.062970 29083 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:32.040936 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2422) took 1260 ms. Trace:
W20260709 13:12:32.060012 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 234420us
I20260709 13:12:32.115927 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:32.116036  1200 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.117077 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:32.119038 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:32.017529  1102 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:12:32.043630  1141 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:32.135967 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.136678 29007 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:32.136828 29083 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:12:32.110960  1212 consensus_meta.cc:287] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.923s	user 0.013s	sys 0.000s
W20260709 13:12:32.144626 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 111797us
I20260709 13:12:32.054143  1263 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:32.148175 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:32.040875  1283 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:32.029080  1119 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.064482  1288 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:32.122623  1262 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:12:32.188650 29002 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:32.035104   932 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.113085 29009 rpcz_store.cc:276] 0709 13:12:30.780129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.854407 (+ 74278us) service_pool.cc:224] Handling call
0709 13:12:30.854765 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:30.854974 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:31.184231 (+329257us) log.cc:844] Serialized 14 byte log entry
0709 13:12:31.185768 (+  1537us) log.cc:844] Serialized 14 byte log entry
0709 13:12:31.867525 (+681757us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b18003e5540. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f41153f7d70 00007f41153eb293 00007f41153e0cea 00007f41153e123a 00007f411584ba2a 00007f411585b0bf 00007f411585b038
0709 13:12:32.040529 (+173004us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:32.040581 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:32.040636 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.040720 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.040866 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53920,"prepare.queue_time_us":986489,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":87,"raft.queue_time_us":122527,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":9100,"spinlock_wait_cycles":13737728,"thread_start_us":493841,"threads_started":2}
W20260709 13:12:32.194589 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 46552.5us
I20260709 13:12:32.198110  1151 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.190553 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:32.137779  1126 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:32.064747  1193 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.178794  1015 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:12:32.197475  1235 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:12:32.184891  1263 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:32.186468  1141 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:32.208092 29010 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:32.195132  1288 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:32.124697  1279 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.199254  1127 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:32.292517 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 85249.9us
I20260709 13:12:32.207563  1151 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:12:32.300542 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 118755us
I20260709 13:12:32.298296  1288 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:32.208961  1015 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.201686  1221 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
W20260709 13:12:32.302428  1061 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.191s	user 0.012s	sys 0.000s
I20260709 13:12:32.298875   934 raft_consensus.cc:2764] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:32.317111 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:32.316136  1302 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.325229  1285 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:32.325692  1302 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.326447  1121 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.326668  1296 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:32.327178  1296 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:32.327538  1296 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:32.335589  1280 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:12:32.302803  1151 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.304363  1291 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:32.311916  1015 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:32.324321  1260 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:32.315768   828 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.316360  1250 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:32.351136 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2579) took 1136 ms. Trace:
I20260709 13:12:32.330370   669 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.352278 29154 rpcz_store.cc:276] 0709 13:12:31.214978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.549137 (+334159us) service_pool.cc:224] Handling call
0709 13:12:31.565786 (+ 16649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.572643 (+  6857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.572685 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.584788 (+ 12103us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.903437 (+318649us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:31.903478 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.903661 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.350920 (+447259us) raft_consensus.cc:1685] finished
0709 13:12:32.350940 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.351078 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":28845,"thread_start_us":318196,"threads_started":1,"wal-append.queue_time_us":450042}
I20260709 13:12:32.346053  1151 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:32.382277 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 65241.6us
I20260709 13:12:32.383399 28975 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:32.383745  1250 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.384292  1250 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.345885  1102 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:12:32.355126  1302 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.393953  1221 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:32.394140  1292 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:32.394946  1269 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 13:12:32.395364  1291 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:32.395771  1291 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:32.396250  1293 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:12:32.396708  1291 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.396731  1290 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
W20260709 13:12:32.397375  1299 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:12:32.399588   967 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.401089  1221 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:32.402034  1221 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:32.404826  1286 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:12:32.406092  1276 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
W20260709 13:12:32.407218  1282 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:12:32.407781 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2439) took 1041 ms. Trace:
I20260709 13:12:32.407860 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2576) took 1196 ms. Trace:
I20260709 13:12:32.408233 29008 rpcz_store.cc:276] 0709 13:12:31.366137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.394825 (+ 28688us) service_pool.cc:224] Handling call
0709 13:12:31.466761 (+ 71936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.492238 (+ 25477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.492270 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.506950 (+ 14680us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.898964 (+392014us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:31.899005 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.899077 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.407562 (+508485us) raft_consensus.cc:1685] finished
0709 13:12:32.407580 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.407728 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10435,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":7440,"thread_start_us":355551,"threads_started":1,"wal-append.queue_time_us":573835}
I20260709 13:12:32.363355   828 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:32.408349 29153 rpcz_store.cc:276] 0709 13:12:31.211591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.211865 (+   274us) service_pool.cc:224] Handling call
0709 13:12:31.287643 (+ 75778us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.426939 (+139296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.426969 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.427413 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.724161 (+296748us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:31.724201 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.724270 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.406988 (+682718us) raft_consensus.cc:1685] finished
0709 13:12:32.407008 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.407611 (+   603us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b5400054e50. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:32.407799 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64398,"raft.run_cpu_time_us":6273,"raft.run_wall_time_us":40407,"spinlock_wait_cycles":39095936,"thread_start_us":220924,"threads_started":1,"wal-append.queue_time_us":485082}
W20260709 13:12:32.415398 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 296400us
I20260709 13:12:32.364899   934 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:32.364214  1301 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:32.355856   669 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:32.418991 29048 connection.cc:256] RPC call timeout handler was delayed by -1.3629604420000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 13:12:32.428227  1301 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:32.444458 29048 connection.cc:256] RPC call timeout handler was delayed by -1.1182541360001323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 13:12:32.445124  1278 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
W20260709 13:12:32.446123 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.446290 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2154) took 1231 ms. Trace:
I20260709 13:12:32.446683 29084 rpcz_store.cc:276] 0709 13:12:31.214775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.214880 (+   105us) service_pool.cc:224] Handling call
0709 13:12:31.306494 (+ 91614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.495282 (+188788us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:31.495355 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.531260 (+ 35905us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.756465 (+225205us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:31.756739 (+   274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.757214 (+   475us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.446076 (+688862us) raft_consensus.cc:1685] finished
0709 13:12:32.446094 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.446233 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23445,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":1617,"thread_start_us":204216,"threads_started":1,"wal-append.queue_time_us":364150}
I20260709 13:12:32.448360  1301 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.429507  1250 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.437563  1123 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.439216  1015 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.440305  1231 raft_consensus.cc:3055] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:32.449395   828 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:32.449522  1231 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:32.450291  1287 log.cc:927] Time spent T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 13:12:32.357702  1141 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.450551  1231 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.440850 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2440) took 1059 ms. Trace:
I20260709 13:12:32.453750 29076 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:32.447774  1240 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:32.451858 29005 rpcz_store.cc:276] 0709 13:12:31.381532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.503945 (+122413us) service_pool.cc:224] Handling call
0709 13:12:31.529371 (+ 25426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.535353 (+  5982us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.535399 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.560686 (+ 25287us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.883462 (+322776us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:31.883503 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.883584 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.440634 (+557050us) raft_consensus.cc:1685] finished
0709 13:12:32.440657 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.440792 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1098,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":4755,"thread_start_us":229340,"threads_started":1,"wal-append.queue_time_us":776513}
W20260709 13:12:32.460940 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 258615us
I20260709 13:12:32.462828 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:32.465977 29048 leader_election.cc:400] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14.
I20260709 13:12:32.466284 29048 leader_election.cc:403] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:32.466944 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.454521  1208 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:32.469056 29048 leader_election.cc:400] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:32.470026 29048 leader_election.cc:403] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:32.470446 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:32.470973 29717 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:32.472472  1298 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:12:32.476800  1208 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.462525   932 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.478839  1312 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:32.480558  1193 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.462092   897 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.455394   934 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.466467  1119 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.517309   934 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.519820   897 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.504022  1285 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.521750  1285 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.523456 29077 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:12:32.523820 29077 pending_rounds.cc:85] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 5
I20260709 13:12:32.524070 29077 pending_rounds.cc:107] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:12:32.524286 29077 raft_consensus.cc:2889] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:32.525466 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:32.525987 29077 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
W20260709 13:12:32.526383 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.526821 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:32.527410 29077 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:32.528492 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:32.529176 29084 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:32.530107 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:32.530834 29080 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:32.532061 29085 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:32.533210 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:32.534016 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 2157) took 1313 ms. Trace:
I20260709 13:12:32.534397 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:32.534482 29076 rpcz_store.cc:276] 0709 13:12:31.220495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.220600 (+   105us) service_pool.cc:224] Handling call
0709 13:12:32.533789 (+1313189us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b5400336654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:32.533923 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2121131392}
I20260709 13:12:32.535142 29083 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:32.536443 29079 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:32.537721   669 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:32.537890 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.538769 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.538980 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.539140  1117 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:32.539558  1117 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:32.539712 29715 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 14: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:32.539836 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.540752 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:32.541078 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.541316 29715 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.542480 29710 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:12:32.544843 29708 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:32.539732  1208 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.546905  1222 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.548188 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.548916 29005 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:32.551595 29715 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
W20260709 13:12:32.526837 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.554131 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.476277  1297 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:12:32.492005  1303 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.101s	user 0.002s	sys 0.002s
I20260709 13:12:32.494081 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.494405 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.495167 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:32.559139 29711 raft_consensus.cc:2410] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:12:32.495357 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.496599  1015 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.497167  1279 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:32.478272  1214 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:12:32.501441  1263 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.502416 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1698) took 1559 ms. Trace:
I20260709 13:12:32.503051 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2575) took 1291 ms. Trace:
I20260709 13:12:32.503548 29717 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:32.508423  1288 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.450548   828 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.555362 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:32.554999 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:32.561353 29081 rpcz_store.cc:276] 0709 13:12:30.942755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.942902 (+   147us) service_pool.cc:224] Handling call
0709 13:12:30.995292 (+ 52390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.901643 (+906351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.502200 (+600557us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54000c8fd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:32.502362 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998892928}
I20260709 13:12:32.562083 29152 rpcz_store.cc:276] 0709 13:12:31.211281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.211774 (+   493us) service_pool.cc:224] Handling call
0709 13:12:32.502904 (+1291130us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003f75d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:32.502998 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340580736}
I20260709 13:12:32.563637 29708 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:32.565515 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.576922 29077 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:32.566639 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:32.575381  1223 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.578190  1309 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.578768  1309 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.580973  1263 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.581151  1309 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.572517  1123 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:32.581941  1123 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.584125  1288 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:32.584348 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.567443 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.572727 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.586022 29150 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:32.572620 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:32.566360 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.577987 29001 raft_consensus.cc:2368] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
I20260709 13:12:32.589648 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.577970  1223 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.577098 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.592703 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:32.593338 29003 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:12:32.556633   934 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:32.601032 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:32.562630   669 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:32.507339  1296 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.583876  1123 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:32.561033  1118 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:12:32.767887 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.677525 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:32.762043 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:32.679066 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:32.953011 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.084164 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:33.084877 29007 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:32.632896 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:32.680423 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.086673 29711 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 10: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:33.086923 29159 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:32.624033 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:33.082526 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 478197us
I20260709 13:12:33.080299 32570 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:32.832798 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 263320us
I20260709 13:12:33.082260 29155 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:32.573009   932 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.704655 29714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 664) took 1531 ms. Trace:
I20260709 13:12:32.713985 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:33.131023 32570 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:33.132076 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:33.132844 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:33.143445 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:33.147768 32570 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:33.148152 32570 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:33.148459 32570 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:33.144429 29714 rpcz_store.cc:276] 0709 13:12:31.172927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.173046 (+   119us) service_pool.cc:224] Handling call
0709 13:12:31.209019 (+ 35973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.220727 (+ 11708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.704302 (+1483575us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b5400576f54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:32.704572 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11126016}
W20260709 13:12:32.632803  1303 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:12:33.153009 29003 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:33.154876 29009 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:33.156973 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:33.159317 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.159688 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:33.160010 29158 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:33.160967 29003 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:12:32.680868  1291 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:32.851262  1310 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.164642 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.161680 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:32.851244  1279 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.165611 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.164580 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.165494 29158 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:32.605332  1288 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:33.171404 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:33.169433 29002 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:33.143831  1223 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:32.814338  1015 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:33.128638  1063 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:32.831756  1221 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:33.182106 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:33.182549  1221 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.184094  1288 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.153645   932 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.174685  1279 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:32.861913  1155 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:33.150170  1297 consensus_meta.cc:287] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.000s
I20260709 13:12:32.615895   996 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.560s
I20260709 13:12:32.593257  1117 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.213783 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2161) took 1915 ms. Trace:
I20260709 13:12:33.181320  1063 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.172751 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:33.220114 29715 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 16: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:33.196415 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:32.624035   934 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:32.633085  1151 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.200008 29155 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 16.
I20260709 13:12:33.136858  1222 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:12:33.229641  1288 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:33.135632  1118 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.188726  1223 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:12:33.258354   912 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:12:33.151026   669 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.265321 29153 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 15: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:33.208642  1297 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:33.166407  1123 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 13:12:33.262982 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.259091   912 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:33.263686 29711 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:33.214279 29082 rpcz_store.cc:276] 0709 13:12:31.298033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.369958 (+ 71925us) service_pool.cc:224] Handling call
0709 13:12:31.731219 (+361261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.732330 (+  1111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.213537 (+1481207us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b540025ab50. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:33.213712 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":829864320}
I20260709 13:12:33.267917 29010 raft_consensus.cc:2410] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:12:33.171914  1310 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.223971   523 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:32.615610 29151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:33.277508 29151 raft_consensus.cc:606] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:33.204332  1155 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.292476 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.293268 29157 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:33.300822 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:33.301571 29005 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:33.304632  1279 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:33.311474 29157 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:12:33.283164   523 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.331897 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.310563  1221 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.203346  1015 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.330595  1123 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.289502 29007 raft_consensus.cc:2410] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:12:33.358615 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:33.360877 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:33.361464 29085 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:33.363222 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.364261 29157 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 10: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:33.362918 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.370154 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:33.371069 29084 raft_consensus.cc:2393] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:33.371876  1310 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.371980 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:33.375725 29156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.016s	sys 0.034s
I20260709 13:12:33.376024 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:33.376236 32570 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:33.376111 29156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.017s	sys 0.034s
I20260709 13:12:33.390815  1015 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:33.392354 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:33.374099  1279 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.396672  1279 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.397014  1015 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:33.398198  1214 consensus_meta.cc:287] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.825s	user 0.011s	sys 0.000s
I20260709 13:12:33.396777 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:33.397930  1015 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.398715 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:33.413806 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:33.377573  1296 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.374610  1297 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:33.427680 32570 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:33.414592 29084 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
W20260709 13:12:33.412470 29710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.013s	sys 0.075s
I20260709 13:12:33.398738  1279 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:33.426312 29150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.069s	sys 0.112s
W20260709 13:12:33.523515 29150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.069s	sys 0.113s
I20260709 13:12:33.362882  1123 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.410557  1214 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14.
I20260709 13:12:33.401352   932 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:33.410288  1117 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:33.484447 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:33.447228  1304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.011s	sys 0.043s
I20260709 13:12:33.487958 29081 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
W20260709 13:12:33.489051 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 408170us
I20260709 13:12:33.472512  1118 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:33.553668 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:33.512787 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 911742us
W20260709 13:12:33.514076 29710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.013s	sys 0.075s
I20260709 13:12:33.480562   669 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:33.527148 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 370219us
I20260709 13:12:33.529513 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:33.534224  1260 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:33.537649 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.554994 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:33.475522  1297 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.559176 29048 leader_election.cc:400] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
I20260709 13:12:33.559185 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:33.561030 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:33.562651 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:33.523203  1296 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:33.563318 29081 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:33.553453  1304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.012s	sys 0.043s
I20260709 13:12:33.572576 29159 raft_consensus.cc:1240] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:33.563731 29050 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:33.574733 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.563938 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:33.537106  1279 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.563562 29048 leader_election.cc:403] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:12:33.578600 29050 leader_election.cc:400] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
I20260709 13:12:33.553844  1221 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:33.562326  1260 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:33.602331 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:33.572943   669 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:33.625157 29159 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:33.629688 32570 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:12:33.647437 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45291.8us
I20260709 13:12:33.573513   523 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:33.647949   669 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:33.649777 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:33.649899 29081 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
W20260709 13:12:33.650149 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 106289us
I20260709 13:12:33.597848  1155 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:33.655323 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:33.595263  1279 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:33.658578 28892 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 06ae4197fd784525b695a9b16c8094ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:33.667834 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:33.659844   988 raft_consensus.cc:993] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:12:33.660494  1214 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:33.664211 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:33.671597 29004 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:33.577057  1117 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
W20260709 13:12:33.668349 28892 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 89458.3us
I20260709 13:12:33.600823  1222 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:12:33.576804  1296 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.628116  1304 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:33.597081  1124 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:12:33.627522  1063 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:33.678174 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:33.680116 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:33.680886 29007 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:33.687290 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:33.687887 29081 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:33.625809   932 consensus_queue.cc:1059] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:12:33.656255   669 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:33.694404 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 69396.3us
I20260709 13:12:33.695382 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:33.670949   912 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.702442 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:33.704298 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:33.651001  1221 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.676710  1155 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:33.686823   988 raft_consensus.cc:1081] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Signalling peer befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:12:33.675264  1214 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:33.707420 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:33.689101  1338 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 10. Current term is 11.
I20260709 13:12:33.724270 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:33.720324 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:33.707159  1113 raft_consensus.cc:3055] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:33.712559  1015 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:33.716323  1326 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:33.727082 29081 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 12: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:33.729293 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:33.729039  1325 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:12:33.718833  1214 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:33.707742  1345 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:33.712885  1342 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:33.718221  1328 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:33.723482  1063 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:33.723070  1118 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:33.686890  1304 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.730886  1113 raft_consensus.cc:740] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:33.747938  1321 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:12:33.748262  1325 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:33.773488  1325 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:33.774727  1304 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.740298  1123 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:33.774648  1325 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.775584  1123 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:33.776374 29710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 667) took 1239 ms. Trace:
I20260709 13:12:33.755286 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 802) took 2603 ms. Trace:
I20260709 13:12:33.776523  1325 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:33.776895 29006 rpcz_store.cc:276] 0709 13:12:31.139144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.139298 (+   154us) service_pool.cc:224] Handling call
0709 13:12:31.139631 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.165117 (+ 25486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.165161 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.167093 (+  1932us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.633108 (+466015us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:31.633153 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.733392 (+2100239us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b540035ded4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:33.733459 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:33.733515 (+    56us) raft_consensus.cc:1685] finished
0709 13:12:33.733536 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:33.742937 (+  9401us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1593,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":2524,"spinlock_wait_cycles":57596160,"thread_start_us":424015,"threads_started":1,"wal-append.queue_time_us":642577,"wal-append.run_cpu_time_us":2938,"wal-append.run_wall_time_us":1568583}
I20260709 13:12:33.719498  1155 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.727677   932 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:33.781169   932 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.781538   932 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:33.782557  1155 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.760140  1345 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.718050  1335 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:12:33.725656  1341 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:33.708519   912 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.762681  1297 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.764283  1124 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:33.770908  1279 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.771090  1113 consensus_queue.cc:260] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.12, Last appended by leader: 12, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.776752 29710 rpcz_store.cc:276] 0709 13:12:32.537159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.543307 (+  6148us) service_pool.cc:224] Handling call
0709 13:12:32.543615 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:32.543958 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:32.543983 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:32.574241 (+ 30258us) log.cc:844] Serialized 26 byte log entry
0709 13:12:33.554871 (+980630us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:33.554908 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.554968 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:33.776166 (+221198us) raft_consensus.cc:1685] finished
0709 13:12:33.776184 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:33.776321 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":287,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":851175,"threads_started":1,"wal-append.queue_time_us":1006922}
I20260709 13:12:33.760309   934 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:33.790396 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 67788.6us
I20260709 13:12:33.722328  1222 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:12:33.741004  1326 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.744251  1015 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.741741  1340 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:33.826380 28975 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:33.827682 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:33.764528  1328 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:33.829782 29150 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
W20260709 13:12:33.797386  1291 consensus_meta.cc:287] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.000s
I20260709 13:12:33.769563  1118 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:33.832476 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:33.841498 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 171893us
W20260709 13:12:33.846887 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.795909  1063 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:33.787601  1342 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:33.797386  1304 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:33.855528 29682 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:33.856312 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 109575us
I20260709 13:12:33.858764 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:33.835253  1113 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:33.878880 29081 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:33.795313   909 consensus_queue.cc:1059] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:12:33.833321  1326 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:33.845062  1340 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.828342  1261 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:12:33.879830 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:33.950593  1362 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.168076 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 288273us
I20260709 13:12:34.168201  1362 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.169338 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:34.170450  1362 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:33.907097 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51659.8us
I20260709 13:12:33.901644 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:33.948410 29680 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:33.884518 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:33.947121   669 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.859215  1118 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:34.500236  1346 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:34.505764  1346 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:33.942260   912 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:33.932206  1221 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:33.849615   934 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:33.933835   909 raft_consensus.cc:3055] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:34.517413   909 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:34.518486 29048 leader_election.cc:336] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.101s (SENT)
I20260709 13:12:34.518388   909 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.518985 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:34.519622   909 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:33.919595  1155 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:34.520924  1155 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.521318  1155 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:33.878958  1124 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:33.901042 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:33.912607  1063 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.844802  1015 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.532543  1015 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.971998   932 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.950528  1214 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.537550   932 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:33.873133  1350 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:12:33.931427  1345 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:33.924391  1333 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:12:34.560191   934 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:33.869349  1341 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.899047  1340 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:33.922819  1353 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.563738  1353 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.565871  1353 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.892220  1296 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:33.957082  1260 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.875564  1342 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:33.880421  1304 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.849046  1328 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:33.913378  1310 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:33.970443  1326 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.054133 29081 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:34.085040  1363 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.089931  1193 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.137075 29154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.000s	sys 0.027s
I20260709 13:12:34.161897 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.170634 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:34.197099 29082 consensus_meta.cc:287] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.002s
I20260709 13:12:34.216181  1222 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:34.230373 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 374533us
I20260709 13:12:34.234031 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.235721 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.254127  1151 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:34.258960 29010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2571) took 1692 ms (client timeout 2101 ms). Trace:
I20260709 13:12:34.256261  1102 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:12:34.262267  1316 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.734s	user 0.004s	sys 0.000s
W20260709 13:12:34.263605  1323 log.cc:927] Time spent T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.731s	user 0.003s	sys 0.000s
I20260709 13:12:34.266005 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:34.279973  1349 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:34.287607 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1886) took 1018 ms. Trace:
I20260709 13:12:34.294601 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:34.320528  1357 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.330686  1339 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:34.335815  1273 log.cc:927] Time spent T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.801s	user 0.001s	sys 0.002s
W20260709 13:12:34.340557  1327 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.684s	user 0.003s	sys 0.000s
W20260709 13:12:34.342017  1315 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.744s	user 0.001s	sys 0.001s
I20260709 13:12:34.344481 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2450) took 2158 ms. Trace:
I20260709 13:12:34.350878 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1881) took 1219 ms. Trace:
W20260709 13:12:34.352735 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.353554 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:34.353788  1325 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.353588 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:34.354664 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:34.354846 29004 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:12:34.359223  1332 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.698s	user 0.001s	sys 0.002s
W20260709 13:12:34.360574  1334 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.758s	user 0.003s	sys 0.000s
I20260709 13:12:34.362825  1117 raft_consensus.cc:3055] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:12:34.364974  1320 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.710s	user 0.004s	sys 0.000s
I20260709 13:12:34.368675 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1704) took 1012 ms. Trace:
W20260709 13:12:34.369926 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 794894us
I20260709 13:12:34.374395  1285 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:34.389235 29682 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:34.389912 29708 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:34.404340  1336 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:34.405547  1223 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:12:34.407445 29008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.007s	sys 0.018s
I20260709 13:12:34.422176  1347 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:34.429273  1261 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.431449 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 345564us
W20260709 13:12:34.442680 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.448355   988 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:34.455631 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.458220 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1706) took 1101 ms. Trace:
W20260709 13:12:34.461905  1317 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.976s	user 0.004s	sys 0.000s
I20260709 13:12:34.467818  1231 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:34.469794 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 521460us
I20260709 13:12:34.469890   897 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:34.475912  1279 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.479228   986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.018s	sys 0.019s
I20260709 13:12:34.480423 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2171) took 2818 ms. Trace:
I20260709 13:12:34.500208  1354 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:34.500727  1288 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.504513  1263 consensus_queue.cc:1059] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:34.508455  1292 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:34.509279 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:34.507865  1346 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:34.524781  1124 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:34.529594  1324 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.535754  1015 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.537995  1214 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:34.545624  1358 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.548477   669 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:34.562747  1341 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.811158 29001 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
W20260709 13:12:34.811991 29154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.000s	sys 0.034s
W20260709 13:12:34.812292 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 423107us
I20260709 13:12:34.812696 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:34.812945 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1882) took 1665 ms. Trace:
I20260709 13:12:34.813802 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:34.811189 29082 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:34.814893 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 819) took 1424 ms. Trace:
I20260709 13:12:34.811002 29155 rpcz_store.cc:276] 0709 13:12:33.269378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.269566 (+   188us) service_pool.cc:224] Handling call
0709 13:12:33.271014 (+  1448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:33.273663 (+  2649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:33.273703 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:33.273754 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:33.273802 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.287355 (+1013553us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b54003bb0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:34.287404 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.287548 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5546880}
W20260709 13:12:34.815582 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 644934us
I20260709 13:12:34.818029 29156 raft_consensus.cc:1240] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: [10.12-10.12]
W20260709 13:12:34.818722 29008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.011s	sys 0.022s
I20260709 13:12:34.818434 29153 rpcz_store.cc:276] 0709 13:12:33.135890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.170203 (+ 34313us) service_pool.cc:224] Handling call
0709 13:12:34.350678 (+1180475us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b540013c254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:34.350817 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92392704}
I20260709 13:12:34.819473 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2578) took 2251 ms. Trace:
I20260709 13:12:34.815676 29081 rpcz_store.cc:276] 0709 13:12:33.390340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.841602 (+451262us) service_pool.cc:224] Handling call
0709 13:12:34.814837 (+973235us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:34.812076 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 302838us
I20260709 13:12:34.820781 29009 rpcz_store.cc:276] 0709 13:12:32.185688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.205226 (+ 19538us) service_pool.cc:224] Handling call
0709 13:12:32.294463 (+ 89237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:33.155382 (+860919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.344199 (+1188817us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54001856d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:34.344416 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978288384}
I20260709 13:12:34.821399 29085 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:34.821938 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1719) took 1276 ms. Trace:
I20260709 13:12:34.822620 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:34.822911 29050 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:34.819900 29008 rpcz_store.cc:276] 0709 13:12:32.568351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.554688 (+986337us) service_pool.cc:224] Handling call
0709 13:12:33.555123 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:33.648668 (+ 93545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:33.648705 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:33.649185 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 13:12:34.819181 (+1169996us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:34.819218 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.819277 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.819316 (+    39us) raft_consensus.cc:1685] finished
0709 13:12:34.819327 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.819425 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":65443,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":20342,"thread_start_us":1190631,"threads_started":2,"wal-append.queue_time_us":274020}
I20260709 13:12:34.828066 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.812098 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:34.828758 29157 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 8.
W20260709 13:12:34.814105 29010 rpcz_store.cc:269] 0709 13:12:32.566044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.566159 (+   115us) service_pool.cc:224] Handling call
0709 13:12:34.258578 (+1692419us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b540022a454. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:34.258911 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561542016}
I20260709 13:12:34.829573 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 829) took 1201 ms. Trace:
I20260709 13:12:34.829909 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:34.831579 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.832851 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:34.830024 29077 rpcz_store.cc:276] 0709 13:12:33.628507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.235489 (+606982us) service_pool.cc:224] Handling call
0709 13:12:34.829518 (+594029us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.830101 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:34.813416 29154 rpcz_store.cc:276] 0709 13:12:33.147443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.171438 (+ 23995us) service_pool.cc:224] Handling call
0709 13:12:33.175368 (+  3930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:33.524015 (+348647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:33.524202 (+   187us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:33.527127 (+  2925us) log.cc:844] Serialized 26 byte log entry
0709 13:12:34.812550 (+1285423us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:34.812647 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.812737 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.812780 (+    43us) raft_consensus.cc:1685] finished
0709 13:12:34.812791 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.812899 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7654,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":1590188,"threads_started":2,"wal-append.queue_time_us":386525}
I20260709 13:12:34.834045 29003 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:34.823773 29083 rpcz_store.cc:276] 0709 13:12:33.356504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.359855 (+  3351us) service_pool.cc:224] Handling call
0709 13:12:33.360217 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:33.507530 (+147313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:33.507578 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:33.511731 (+  4153us) log.cc:844] Serialized 26 byte log entry
0709 13:12:33.672067 (+160336us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:33.672134 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.672220 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.368411 (+696191us) raft_consensus.cc:1685] finished
0709 13:12:34.368438 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.368612 (+   174us) inbound_call.cc:177] Queueing success response
0709 13:12:34.529262 (+160650us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b480015c258. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411a76aaa1 00007f411a761b37 00007f411a7619ca 00007f411a85926f 00007f411a858f37 00007f411a86d55c 00007f411a86d508 00007f411a86d488 00007f411a86d443 00007f411a86c6ec 00007f4119796a96
Metrics: {"mutex_wait_us":7925,"raft.queue_time_us":200225,"raft.run_cpu_time_us":5213,"raft.run_wall_time_us":963848,"spinlock_wait_cycles":17372544,"thread_start_us":238874,"threads_started":2,"wal-append.queue_time_us":279005}
I20260709 13:12:34.832944 29004 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:34.835497 29150 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:34.822238 29085 rpcz_store.cc:276] 0709 13:12:33.545276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.161626 (+616350us) service_pool.cc:224] Handling call
0709 13:12:34.821885 (+660259us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:34.838047 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 369343us
I20260709 13:12:34.838902 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:34.839504 29081 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:34.839640 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:34.840502 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.814723 29079 rpcz_store.cc:276] 0709 13:12:31.661641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.881306 (+219665us) service_pool.cc:224] Handling call
0709 13:12:32.536148 (+654842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:32.536769 (+   621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.480180 (+1943411us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b5400336650. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:34.480335 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505052032}
I20260709 13:12:34.810981  1223 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:34.845360 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.845717 29010 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:34.828117   996 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:12:34.846987 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.849089 29081 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:34.863530 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:34.855674 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:34.869132 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.870616 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:34.854748 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1701) took 1499 ms. Trace:
I20260709 13:12:34.857149 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.882375 29082 rpcz_store.cc:276] 0709 13:12:33.355630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.355793 (+   163us) service_pool.cc:224] Handling call
0709 13:12:34.854371 (+1498578us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540034e754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:34.854665 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422980864}
I20260709 13:12:34.849781 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1720) took 1304 ms. Trace:
I20260709 13:12:34.872264 28976 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:34.883801 29081 rpcz_store.cc:276] 0709 13:12:33.545542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.838637 (+1293095us) service_pool.cc:224] Handling call
0709 13:12:34.849728 (+ 11091us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.858543 29680 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:34.849964 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 828) took 1278 ms. Trace:
I20260709 13:12:34.885123 29080 rpcz_store.cc:276] 0709 13:12:33.571330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.233746 (+662416us) service_pool.cc:224] Handling call
0709 13:12:34.849891 (+616145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.885501 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.886082 29156 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:34.886920 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:34.826090   986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.166s	user 0.022s	sys 0.024s
I20260709 13:12:34.881258 29709 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:34.882009 29716 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 7.
I20260709 13:12:34.892557 29084 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:34.893594 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 825) took 1501 ms. Trace:
I20260709 13:12:34.893352 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:34.894711 29085 rpcz_store.cc:276] 0709 13:12:33.391828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.868887 (+1477059us) service_pool.cc:224] Handling call
0709 13:12:34.893539 (+ 24652us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.878794 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.893214 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.823235 29078 rpcz_store.cc:276] 0709 13:12:33.357030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.362092 (+  5062us) service_pool.cc:224] Handling call
0709 13:12:33.379077 (+ 16985us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:33.564335 (+185258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:33.564398 (+    63us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:33.576941 (+ 12543us) log.cc:844] Serialized 52 byte log entry
0709 13:12:34.437181 (+860240us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b180011e9c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:12:34.457371 (+ 20190us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:34.457654 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:12:34.457838 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.457911 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.457951 (+    40us) raft_consensus.cc:1685] finished
0709 13:12:34.458032 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.458168 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49014,"prepare.queue_time_us":5038,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":267711,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":13317,"spinlock_wait_cycles":13475712,"thread_start_us":154319,"threads_started":2,"wal-append.queue_time_us":833994}
I20260709 13:12:34.898015 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:34.897964 29079 raft_consensus.cc:2393] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:34.899614 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39900 (request call id 822) took 1508 ms. Trace:
I20260709 13:12:34.898638 29085 raft_consensus.cc:2376] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12. Re-sending same reply.
I20260709 13:12:34.900243 29079 rpcz_store.cc:276] 0709 13:12:33.391085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.878477 (+1487392us) service_pool.cc:224] Handling call
0709 13:12:34.899554 (+ 21077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.901049 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.901762 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:34.902139 29085 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:12:34.901351 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.904008 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 814) took 1514 ms. Trace:
I20260709 13:12:34.904443 29085 rpcz_store.cc:276] 0709 13:12:33.389175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.903174 (+1513999us) service_pool.cc:224] Handling call
0709 13:12:34.903423 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.903721 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.903757 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:34.903795 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:34.903825 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.903875 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.903961 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.905992 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 815) took 1516 ms. Trace:
I20260709 13:12:34.906363 29085 rpcz_store.cc:276] 0709 13:12:33.389401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.905324 (+1515923us) service_pool.cc:224] Handling call
0709 13:12:34.905567 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.905719 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.905744 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:34.905780 (+    36us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:34.905807 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.905857 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.905949 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.909175 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:34.894091 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:34.830564  1304 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:34.915303 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.827925   912 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.911350 29711 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:12:34.912297 29077 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:34.909204 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:34.828792   932 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.935519 29084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1714) took 1547 ms (client timeout 1882 ms). Trace:
I20260709 13:12:34.837277   934 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:34.856969 28902 catalog_manager.cc:5697] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 9, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.831002  1117 raft_consensus.cc:740] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:34.916791 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.950670 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1718) took 1405 ms. Trace:
I20260709 13:12:34.918490 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:34.915844 29153 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:12:34.951012 29077 rpcz_store.cc:276] 0709 13:12:33.544967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.840288 (+1295321us) service_pool.cc:224] Handling call
0709 13:12:34.950496 (+110208us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b5400022954. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:34.950614 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144636160}
I20260709 13:12:34.900671 29682 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:34.952431 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51835.8us
I20260709 13:12:34.857623  1222 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:34.924832 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.931704 29051 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:34.846741   988 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.959322  1117 consensus_queue.cc:260] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.960685  1117 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:34.962157   988 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:34.933941  1221 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:34.933511  1124 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.899159  1214 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:34.837430  1353 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:34.837517   909 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:34.572492  1155 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.838858  1306 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:34.570168  1340 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.828244  1350 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.941141 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:34.984597 29078 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:34.940536  1309 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:34.941614 29084 rpcz_store.cc:269] 0709 13:12:33.388331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.536299 (+147968us) service_pool.cc:224] Handling call
0709 13:12:34.935366 (+1399067us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54000465d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:34.935470 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3056973056}
I20260709 13:12:34.943373 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:34.943977 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.946055 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.986737 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.986879 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:34.916168 29156 raft_consensus.cc:2393] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:34.988184 29155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:34.855959  1063 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:34.990749 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 47447.7us
I20260709 13:12:34.840169  1261 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.995131  1113 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:34.846765  1363 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.948292 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 816) took 1558 ms. Trace:
I20260709 13:12:34.950456   669 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:34.948489 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 820) took 1557 ms. Trace:
I20260709 13:12:34.892062  1296 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:34.917829 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:34.857350   996 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.958209  1260 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:34.958477 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1705) took 1601 ms. Trace:
I20260709 13:12:34.958025 28902 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 9, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 9 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:34.959108 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:34.846891  1288 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.959211 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:34.908769  1291 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:34.811076  1336 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:34.966611 29077 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:34.967738 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1722) took 1413 ms. Trace:
I20260709 13:12:34.909754  1345 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:34.571851  1118 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.829752   897 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:34.973619 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:34.975610 29008 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:34.917394  1344 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:34.844436  1279 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.849653  1015 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:34.937250 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.836934  1325 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.840030  1193 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:35.075718 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 102208us
I20260709 13:12:34.934937  1371 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:34.861654  1346 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:12:34.991858 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:34.839897  1358 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:34.990895 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9d20 after lost signal to thread 27901
W20260709 13:12:35.078764 28895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 86948.3us
I20260709 13:12:34.893889  1285 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.994195 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1880) took 2318 ms. Trace:
I20260709 13:12:34.994254 29712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 572) took 2495 ms. Trace:
W20260709 13:12:34.994621 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 76195.8us
I20260709 13:12:34.992401  1063 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:34.847908  1354 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.969303   934 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:35.098037   934 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.847018  1347 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.949620   932 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:34.996968 29006 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:35.048632 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.001276 29082 rpcz_store.cc:276] 0709 13:12:33.389648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.905948 (+1516300us) service_pool.cc:224] Handling call
0709 13:12:34.947818 (+ 41870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.947978 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.948002 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:34.948064 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:34.948095 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.948146 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.948243 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.060127 29710 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:35.060436 29079 rpcz_store.cc:276] 0709 13:12:33.554383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.924496 (+1370113us) service_pool.cc:224] Handling call
0709 13:12:34.967659 (+ 43163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.061074 29076 rpcz_store.cc:276] 0709 13:12:33.356774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.356876 (+   102us) service_pool.cc:224] Handling call
0709 13:12:33.357166 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:33.357466 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:33.357491 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:33.357909 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:12:33.489292 (+131383us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:33.489341 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.489410 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.358727 (+869317us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:34.949776 (+591049us) raft_consensus.cc:1685] finished
0709 13:12:34.949796 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.958417 (+  8621us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":165605,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":27253,"spinlock_wait_cycles":14513920,"thread_start_us":119530,"threads_started":1,"wal-append.queue_time_us":207048}
I20260709 13:12:34.976876  1232 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:35.001564 29080 rpcz_store.cc:276] 0709 13:12:33.390601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.908223 (+1517622us) service_pool.cc:224] Handling call
0709 13:12:34.948082 (+ 39859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.948223 (+   141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.948244 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:34.948278 (+    34us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:34.948312 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.948363 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.948445 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.946257   912 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.073302 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 821) took 1682 ms. Trace:
W20260709 13:12:35.115382 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 174601us
I20260709 13:12:34.963292  1124 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:35.077163 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 219179us
W20260709 13:12:35.079053 29180 debug-util.cc:398] Leaking SignalData structure 0x7b0800053da0 after lost signal to thread 27904
I20260709 13:12:35.085736  1233 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:35.150120 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6160 after lost signal to thread 28942
W20260709 13:12:35.151773 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08000bbd60 after lost signal to thread 29034
I20260709 13:12:35.087188 29712 rpcz_store.cc:276] 0709 13:12:32.494635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.495960 (+  1325us) service_pool.cc:224] Handling call
0709 13:12:32.496326 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:32.496683 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:32.496709 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:32.497140 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.585757 (+ 88617us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:32.585798 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.585862 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.341526 (+1755664us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:34.979165 (+637639us) raft_consensus.cc:1685] finished
0709 13:12:34.979190 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.989896 (+ 10706us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":364,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"spinlock_wait_cycles":167388544,"thread_start_us":88234,"threads_started":1,"wal-append.queue_time_us":1007742}
W20260709 13:12:35.152915 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08000a9520 after lost signal to thread 29035
I20260709 13:12:35.078614  1345 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:35.038975  1349 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:35.062527  1291 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:35.019637   669 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.087710 29683 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:35.159214 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71612us
I20260709 13:12:35.159878 29683 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:35.148833 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:35.160914 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:35.161882 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:34.990502   909 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:35.094398 29152 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:12:34.995972  1341 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:34.990188  1379 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:35.098973 29157 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:35.100495 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 277641us
I20260709 13:12:35.087072 29158 rpcz_store.cc:276] 0709 13:12:32.675974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.176296 (+500322us) service_pool.cc:224] Handling call
0709 13:12:33.181854 (+  5558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:33.182182 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:33.182211 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:33.182645 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 13:12:33.478085 (+295440us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:33.478143 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.478235 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.340051 (+861816us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:34.991386 (+651335us) raft_consensus.cc:1685] finished
0709 13:12:34.991413 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.994113 (+  2700us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1397,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":49,"spinlock_wait_cycles":11269888,"thread_start_us":295045,"threads_started":1,"wal-append.queue_time_us":386699}
I20260709 13:12:35.074195   897 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:35.133695 29077 rpcz_store.cc:276] 0709 13:12:33.390843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.951875 (+1561032us) service_pool.cc:224] Handling call
0709 13:12:34.952161 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.072989 (+120828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.073239 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.079785  1380 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.078186  1371 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.083251  1260 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:34.993746  1353 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:35.154714 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd960 after lost signal to thread 29109
W20260709 13:12:35.215696 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08002ef2c0 after lost signal to thread 29110
I20260709 13:12:35.157157  1233 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:35.216958 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08002a0d60 after lost signal to thread 29183
I20260709 13:12:35.158375  1349 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.095449  1285 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.163846  1113 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:35.082971  1346 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:35.095263  1063 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:35.229436 32570 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:35.210088  1214 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:35.099692  1342 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:35.177529 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 183761us
I20260709 13:12:35.167357 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 827) took 1760 ms. Trace:
I20260709 13:12:35.243132 29080 rpcz_store.cc:276] 0709 13:12:33.407066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.166449 (+1759383us) service_pool.cc:224] Handling call
0709 13:12:35.166808 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.167001 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.167036 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:35.167088 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:35.167132 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.167192 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.167305 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.244695 29080 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:35.221107  1233 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:35.223623  1349 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.208250 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:35.245287 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1717) took 1700 ms. Trace:
W20260709 13:12:35.252897 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 91056.3us
I20260709 13:12:35.253314 29080 rpcz_store.cc:276] 0709 13:12:33.544663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.244090 (+1699427us) service_pool.cc:224] Handling call
0709 13:12:35.244367 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.245061 (+   694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.245238 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.168560 29151 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:12:35.217310 29180 debug-util.cc:398] Leaking SignalData structure 0x7b0800138aa0 after lost signal to thread 29184
I20260709 13:12:35.084210  1288 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:34.990181  1306 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.101673  1222 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:34.983310  1223 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:35.218547 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 281376us
W20260709 13:12:35.265057 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.163765  1291 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:35.221757 29048 leader_election.cc:336] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.523s (SENT)
I20260709 13:12:34.997378  1350 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:35.275214 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.223080 29158 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:35.168175  1244 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:34.996752  1261 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:35.133934  1310 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:12:35.207669   988 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:35.255545  1113 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:35.213001  1260 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.084968  1325 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.212803  1371 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:35.174286 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:35.220389  1326 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:35.248471 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 316807us
W20260709 13:12:35.333284 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.334219 29051 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:35.134703  1232 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:35.256104 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08003fcf20 after lost signal to thread 29741
I20260709 13:12:35.226856  1344 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:35.257804 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 49659.4us
W20260709 13:12:35.338348 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08002c51c0 after lost signal to thread 29742
I20260709 13:12:35.338852 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:35.258543 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:35.339622 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.271250 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:35.341243 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 119521us
I20260709 13:12:35.278736 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:35.228721 32630 consensus_queue.cc:1059] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:12:35.169376  1373 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:12:35.213119  1380 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.266314  1342 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:35.299024 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 137183us
I20260709 13:12:35.292618  1340 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:35.336768 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 162566us
I20260709 13:12:35.248720  1214 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.342896 29158 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:35.368146 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:35.344817 29003 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:35.290625   988 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.219622  1379 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.350000 29050 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:35.375530 29050 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:35.332440  1260 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:35.368986 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2463) took 1099 ms. Trace:
I20260709 13:12:35.341518  1232 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:35.230348  1336 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.342298  1325 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:35.343581  1344 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:35.357717  1113 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:35.397282 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:35.352908  1386 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:35.357368  1326 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.258605  1345 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.373299  1015 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.289572  1346 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.374960  1288 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:35.374818   988 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:35.357384  1381 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:35.401273 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:35.379341 29003 rpcz_store.cc:276] 0709 13:12:34.269422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.829909 (+560487us) service_pool.cc:224] Handling call
0709 13:12:35.368934 (+539025us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.395787  1340 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:35.373591  1354 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.426273 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:35.437482  1340 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:35.438141  1340 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:35.427058 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2452) took 1161 ms. Trace:
I20260709 13:12:35.397536  1310 raft_consensus.cc:3055] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:35.438767  1310 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:35.438609 29007 rpcz_store.cc:276] 0709 13:12:34.265184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.265528 (+   344us) service_pool.cc:224] Handling call
0709 13:12:34.265883 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:34.497614 (+231731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.497651 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:34.498134 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:12:34.832322 (+334188us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:34.832360 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.832421 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:35.425371 (+592950us) raft_consensus.cc:1685] finished
0709 13:12:35.426835 (+  1464us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.426991 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":231352,"raft.queue_time_us":233001,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":333834,"threads_started":1,"wal-append.queue_time_us":447995}
I20260709 13:12:35.440141  1310 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.441689  1310 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:35.281036   909 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.398370  1325 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.398985  1113 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:35.455876 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53527.1us
I20260709 13:12:35.398665  1221 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.341370  1353 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.402329   932 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:35.399400  1358 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:35.432403  1336 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:35.473974  1328 consensus_meta.cc:287] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.573s	user 0.001s	sys 0.008s
I20260709 13:12:35.432740  1381 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:35.432230  1385 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:35.476871 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 122909us
I20260709 13:12:35.508322 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:35.512984 29158 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:35.465626  1325 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:35.455228  1288 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:35.465087  1386 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:35.465217  1113 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:35.406498  1326 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.517221 29158 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:35.477236 32630 raft_consensus.cc:3055] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:12:35.521512  1106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.001s	sys 0.016s
I20260709 13:12:35.479777   912 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:35.530808  1387 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:12:35.530642  1288 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.531913 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 817) took 2142 ms. Trace:
I20260709 13:12:35.532306 29081 rpcz_store.cc:276] 0709 13:12:33.389873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.906610 (+1516737us) service_pool.cc:224] Handling call
0709 13:12:34.906891 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:34.907169 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.907195 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:34.932673 (+ 25478us) log.cc:844] Serialized 26 byte log entry
0709 13:12:35.136204 (+203531us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:35.136260 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.136346 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:35.531700 (+395354us) raft_consensus.cc:1685] finished
0709 13:12:35.531720 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.531856 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99153,"spinlock_wait_cycles":14154624,"thread_start_us":196867,"threads_started":1,"wal-append.queue_time_us":289680}
I20260709 13:12:35.533820 29083 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:35.477852  1221 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.514171  1285 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.515578   669 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:35.512413  1381 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:35.528494  1106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.001s	sys 0.016s
W20260709 13:12:35.611851 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 185639us
I20260709 13:12:35.613340 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:35.640026 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:35.433034  1347 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:35.526599 32630 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:35.592818  1392 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
W20260709 13:12:35.593063  1388 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:12:35.599853  1279 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.640393  1285 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.515403  1328 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.654078 28975 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:35.534722  1385 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:35.547683  1306 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.601049  1363 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:35.671366 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:35.642300  1386 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.679523 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:35.682675  1397 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:12:35.679682  1347 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:35.679769  1279 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:35.679358  1285 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.672817  1413 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.682915  1382 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 13:12:35.696511 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.686678  1397 raft_consensus.cc:3055] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:35.697954  1397 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:35.698721  1397 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.699924  1397 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:35.688647  1380 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.705274  1221 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.705066  1340 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.707471  1340 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.691957 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:35.666596  1350 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:35.708385  1309 consensus_meta.cc:287] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.000s
I20260709 13:12:35.796121 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:35.798166 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 118776us
I20260709 13:12:35.800714 29717 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:35.681105  1292 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
W20260709 13:12:35.811867  1405 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.259s	user 0.000s	sys 0.000s
W20260709 13:12:35.694136  1304 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.003s
I20260709 13:12:35.813369 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 818) took 2423 ms. Trace:
I20260709 13:12:35.694861  1285 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.815840 29085 rpcz_store.cc:276] 0709 13:12:33.390097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.907266 (+1517169us) service_pool.cc:224] Handling call
0709 13:12:34.907530 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.214184 (+306654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.214223 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:35.228312 (+ 14089us) log.cc:844] Serialized 26 byte log entry
0709 13:12:35.708741 (+480429us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:35.708780 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.708859 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:35.813148 (+104289us) raft_consensus.cc:1685] finished
0709 13:12:35.813170 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.813306 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113774,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":11151,"thread_start_us":725322,"threads_started":2,"wal-append.queue_time_us":554111}
I20260709 13:12:35.696949  1214 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:35.820216 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 455491us
I20260709 13:12:35.845057 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:35.822358 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:35.824159  1304 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.824957   988 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.691115  1232 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:35.803063  1117 consensus_meta.cc:287] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.842s	user 0.010s	sys 0.000s
I20260709 13:12:35.822373   996 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:35.828513  1233 consensus_meta.cc:287] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.002s
W20260709 13:12:35.817829  1231 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.987s	user 0.009s	sys 0.003s
I20260709 13:12:35.803507  1244 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:35.856022 29709 raft_consensus.cc:2410] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:12:35.829551   897 consensus_meta.cc:287] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.001s
I20260709 13:12:35.693915  1328 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:35.760280  1344 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.856266 29713 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:35.843935  1378 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:35.802685  1309 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:35.803141  1412 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:35.691455  1347 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:35.692540  1400 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:12:35.760288  1326 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:35.848593 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:35.839483  1113 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.828739 32630 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.855209 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:35.761588  1386 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:35.859372 29050 connection.cc:256] RPC call timeout handler was delayed by -3.1554139079999004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:12:35.859404 29002 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 12. Ops: []
I20260709 13:12:35.692924  1381 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:35.862505  1291 consensus_meta.cc:287] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.000s
I20260709 13:12:35.863024  1232 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.863603 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.858841  1362 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.860636  1117 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.859668   897 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.865653 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.860838  1379 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:35.866448 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.853677  1354 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:35.872165 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:35.693981  1413 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.875550 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.876958 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.877344 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.896517 29076 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:35.897464 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.880370 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.881217 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:35.878664   828 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:35.858922  1285 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:35.891350 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.881248 29715 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:35.896080 29079 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:35.857012  1233 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.888624  1378 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.899917 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.895016 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:35.901185 28976 connection.cc:256] RPC call timeout handler was delayed by -1.9409343680000575s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
I20260709 13:12:35.901695   988 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:35.901805 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.890805  1340 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.898725 29078 raft_consensus.cc:2376] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14. Re-sending same reply.
I20260709 13:12:35.921666  1340 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.922156 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:35.919828  1408 log.cc:927] Time spent T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:12:35.888792  1117 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.911973  1285 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.925632 29085 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:35.881879  1291 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:35.928400  1393 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.636s	user 0.003s	sys 0.000s
I20260709 13:12:35.902420  1296 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:35.929325  1296 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.929946  1340 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.930189 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:35.930735 29078 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:35.931591 29076 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:35.934032 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:35.935326 29710 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:35.936365 29717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 887) took 1078 ms. Trace:
I20260709 13:12:35.937567 29081 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:35.938139 29078 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:35.874476  1347 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:35.915413 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:35.907698  1063 consensus_meta.cc:287] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.664s	user 0.005s	sys 0.005s
I20260709 13:12:35.917040 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.917810  1378 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:35.918838  1260 consensus_meta.cc:287] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.517s	user 0.006s	sys 0.000s
I20260709 13:12:35.884454  1362 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.928867  1285 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:35.909994 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:35.940393 29122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.002s	sys 0.010s
W20260709 13:12:35.951119 29122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.002s	sys 0.010s
I20260709 13:12:35.941478 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.942363 29716 raft_consensus.cc:2410] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 12 because replica has last-logged OpId of term: 10 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 11.
I20260709 13:12:35.951653 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.952031 29080 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:35.936705 29717 rpcz_store.cc:276] 0709 13:12:34.857968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.862701 (+  4733us) service_pool.cc:224] Handling call
0709 13:12:34.863028 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.801133 (+938105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.936161 (+135028us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b540068ecd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:35.936315 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155986304}
I20260709 13:12:35.942955  1426 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:35.915917   897 leader_election.cc:290] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:35.948971 29711 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:35.956400 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.956970 29714 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 9.
I20260709 13:12:35.957720 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.958850 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.959141 29714 raft_consensus.cc:2410] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:12:35.951452 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 660994us
W20260709 13:12:35.911571 28976 connection.cc:256] RPC call timeout handler was delayed by -1.7960343680000506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
I20260709 13:12:35.953635 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:35.925742  1117 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:35.945619  1381 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.958709 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:35.908416  1419 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:35.962111 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:35.960994 28976 connection.cc:256] RPC call timeout handler was delayed by -1.653834368000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
I20260709 13:12:35.971276 29079 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:35.972496 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.973163 29079 raft_consensus.cc:2393] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:35.973487  1386 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.973856 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.974438 29079 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:35.974726  1117 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:35.975721  1429 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:35.976112  1429 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:35.976868  1429 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.977524  1233 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.978003  1429 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:35.975178 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.953068  1260 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.980213  1419 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:35.980593 29079 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:35.968884 29717 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:35.982419  1419 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.984906 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:35.985662  1155 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:35.986707 29076 raft_consensus.cc:2368] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12.
I20260709 13:12:35.969596 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:35.988155 28976 connection.cc:256] RPC call timeout handler was delayed by -1.5197343680000586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
W20260709 13:12:35.988572 28976 connection.cc:256] RPC call timeout handler was delayed by -1.0967343680000567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25730000000000003s for a response
I20260709 13:12:35.989023 29003 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:35.989900 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:35.990957 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:35.991048 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:35.991478 29003 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:35.992213 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:35.992319 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:35.992758 29004 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:35.992898 29010 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:35.993963 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:35.981611 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:35.995028 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:35.944713  1244 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:35.997884 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:35.995113  1124 consensus_meta.cc:287] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.840s	user 0.010s	sys 0.000s
I20260709 13:12:35.998540 29003 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:35.999640 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.000634 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:35.995505 29079 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:36.002050 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.001907 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.003103 29079 raft_consensus.cc:2410] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:12:36.003216 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.004280  1328 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:36.000748 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.005589   909 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.005052 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:36.008893 29008 raft_consensus.cc:1240] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:36.009513 29003 raft_consensus.cc:2410] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:12:36.010818 29717 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:36.011067 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.006609   996 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:36.015488 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:36.016189 29082 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:36.024242   897 consensus_queue.cc:1059] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:36.026466   897 raft_consensus.cc:3055] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:36.026787 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.026880   897 raft_consensus.cc:740] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:36.027472 29085 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
W20260709 13:12:36.027588 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.239s (SENT) [suppressed 10 similar messages]
I20260709 13:12:36.029191 29085 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:36.030417 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 706) took 1145 ms. Trace:
I20260709 13:12:36.031082 29085 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:36.032470  1326 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:36.016161  1113 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:36.033283  1328 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:35.950204  1063 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.033818   996 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:36.033851  1328 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:36.039597  1328 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.040931  1328 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:36.041555 28902 catalog_manager.cc:5697] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.042575 29082 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:36.044837  1354 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:36.044714   996 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:36.045353 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 824) took 2653 ms. Trace:
W20260709 13:12:36.046361 29083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 1721) took 2498 ms (client timeout 2191 ms). Trace:
W20260709 13:12:36.046800 29083 rpcz_store.cc:269] 0709 13:12:33.547776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.838119 (+1290343us) service_pool.cc:224] Handling call
0709 13:12:36.046191 (+1208072us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b5400404554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:36.046297 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1493742464}
I20260709 13:12:36.049206 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 818) took 1221 ms. Trace:
I20260709 13:12:36.049626 29005 rpcz_store.cc:276] 0709 13:12:34.827349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.847402 (+ 20053us) service_pool.cc:224] Handling call
0709 13:12:34.847640 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.159853 (+312213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.159899 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:35.159955 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:35.159999 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.048951 (+888952us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54003e21d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:36.049011 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.049144 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717393664}
I20260709 13:12:36.050393  1118 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.051215 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:36.051800  1404 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
I20260709 13:12:36.039626  1063 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.054500 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:36.055219 29709 raft_consensus.cc:2368] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15.
I20260709 13:12:36.052963 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 820) took 1218 ms. Trace:
I20260709 13:12:36.030840 29159 rpcz_store.cc:276] 0709 13:12:34.885420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.885524 (+   104us) service_pool.cc:224] Handling call
0709 13:12:34.915024 (+ 29500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.964689 (+ 49665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.964717 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:34.964758 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:34.964789 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.030180 (+1065391us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b54003ec6d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:36.030227 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.030356 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113799552}
I20260709 13:12:36.045845 29084 rpcz_store.cc:276] 0709 13:12:33.391576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.038879 (+1647303us) service_pool.cc:224] Handling call
0709 13:12:35.073392 (+ 34513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.401864 (+328472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.401892 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:35.401932 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:35.401960 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.045141 (+643181us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400120a54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:36.045184 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.045303 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754975104}
I20260709 13:12:36.064502 29083 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [8.6-9.7]
W20260709 13:12:35.994905  1371 consensus_meta.cc:287] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.000s
I20260709 13:12:36.080005 29125 leader_election.cc:400] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15.
I20260709 13:12:36.043277 28902 catalog_manager.cc:5697] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 9, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.058878 29002 rpcz_store.cc:276] 0709 13:12:34.834286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.898980 (+ 64694us) service_pool.cc:224] Handling call
0709 13:12:34.899264 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.872028 (+972764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.052751 (+180723us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54001caf54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:36.052909 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2207645056}
I20260709 13:12:36.080428 29125 leader_election.cc:403] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:12:36.084738 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.089006 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:36.089808 29709 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:36.055238  1415 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.370s	user 0.002s	sys 0.000s
I20260709 13:12:36.091444 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.096261 29008 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:35.982436  1260 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.006134   909 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:36.093794  1376 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:36.036758  1193 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.102974 29717 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:36.027910   897 consensus_queue.cc:260] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.119755   897 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:36.131661 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.134888 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.138774 29008 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:36.137589  1288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.011s
I20260709 13:12:36.151162 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.152575 29003 raft_consensus.cc:2410] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 12 because replica has last-logged OpId of term: 10 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 11.
I20260709 13:12:36.087549  1371 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:36.165599  1413 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:36.170885 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:36.149286  1288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.006s	sys 0.012s
I20260709 13:12:36.176343 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:36.177040 29083 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:36.178380 29004 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:36.179395 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.180256 29004 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:36.181052 29009 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:12:36.174480 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.191545 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:36.192127 29003 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:36.188836  1325 consensus_meta.cc:287] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.663s	user 0.004s	sys 0.007s
I20260709 13:12:36.197973 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:36.232697  1431 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:12:36.236850 29151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1910) took 1348 ms. Trace:
I20260709 13:12:36.237134 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 679) took 1243 ms. Trace:
W20260709 13:12:36.238458 29006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2451) took 1973 ms (client timeout 1510 ms). Trace:
W20260709 13:12:36.238915 29006 rpcz_store.cc:269] 0709 13:12:34.264932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.353168 (+ 88236us) service_pool.cc:224] Handling call
0709 13:12:36.224816 (+1871648us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54000bf4d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:36.238391 (+ 13575us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415353088}
I20260709 13:12:36.240023 29709 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:36.237304 29151 rpcz_store.cc:276] 0709 13:12:34.888051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.084066 (+196015us) service_pool.cc:224] Handling call
0709 13:12:35.084384 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.254585 (+170201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.225098 (+970513us) spinlock_profiling.cc:243] Waited 79.9 ms on lock 0x7b54003edfd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:36.236788 (+ 11690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183844736}
I20260709 13:12:36.243939 29083 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:36.245261 29078 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:36.237632 29708 rpcz_store.cc:276] 0709 13:12:34.993322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.037477 (+ 44155us) service_pool.cc:224] Handling call
0709 13:12:35.073393 (+ 35916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.271111 (+197718us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:12:35.403507 (+132396us) log.cc:844] Serialized 14 byte log entry
0709 13:12:35.799320 (+395813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:35.799398 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:35.799464 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.227415 (+427951us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54001f0754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:36.236927 (+  9512us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.237062 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":86539,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":4439,"raft.queue_time_us":2872,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7643,"spinlock_wait_cycles":454047616,"thread_start_us":509715,"threads_started":2,"wal-append.queue_time_us":390177}
I20260709 13:12:36.242041 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:36.246127 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.254542 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:36.269497 29077 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:36.273603 29077 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:12:36.279743 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:36.241665  1412 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:36.285809 29077 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:36.293990 29077 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:12:36.296540 29122 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:36.297268 32570 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:36.297704 32570 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:36.286105 29003 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:36.298372 32570 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:36.299917 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:36.308993 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:36.281975  1193 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:36.244021  1325 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.310035  1326 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:36.314133  1118 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:36.290223  1426 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:36.338841  1426 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.339294  1426 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:36.343305 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:36.301558  1412 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:36.344213 29710 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 15: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:36.319947 29083 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:36.345346 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:36.346362 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.346853 29083 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:36.323773 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.327795  1371 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:36.328600  1378 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.329684 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:36.348896 29708 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:36.337339 29079 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 15: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:36.338294  1118 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:36.350287  1118 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:36.348234  1436 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:12:36.341997  1325 leader_election.cc:290] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:36.363389 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.362106 29082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:44428
W20260709 13:12:36.374950  1433 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:12:36.389703 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 699) took 1571 ms. Trace:
I20260709 13:12:36.390137 29152 rpcz_store.cc:276] 0709 13:12:34.818057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.818841 (+   784us) service_pool.cc:224] Handling call
0709 13:12:36.389470 (+1570629us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54003f66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:36.389631 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571150080}
W20260709 13:12:36.397099 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.359109  1347 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.431232 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.432013 29159 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:36.433475 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.437058 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.440699 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.441496 29150 raft_consensus.cc:2410] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:12:36.443737 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.447435 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.448526 29159 raft_consensus.cc:2410] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:12:36.452247 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.457162 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:36.416254  1349 consensus_meta.cc:287] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.149s	user 0.013s	sys 0.008s
I20260709 13:12:36.401809  1328 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:36.464520 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.466363 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.459347  1376 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15.
I20260709 13:12:36.471565 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.419360  1325 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:36.495771 29151 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:36.460071  1378 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:36.508108 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.509775 29150 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:36.469106  1349 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:36.508178  1432 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.112s	user 0.000s	sys 0.002s
W20260709 13:12:36.508522  1429 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.003s
I20260709 13:12:36.496420  1326 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:36.526475 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.523396 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.531371 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.532066 29152 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:36.507330  1325 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.507987  1378 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.543491 29156 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:36.507146  1328 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.544667  1260 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.545423  1260 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.548401  1260 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.549511 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:36.550132 29708 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:36.552335 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:36.539306  1417 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.892s	user 0.004s	sys 0.000s
I20260709 13:12:36.559165 29159 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:36.559630 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:36.558158  1439 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:12:36.568584 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:36.573009  1440 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:12:36.561478  1347 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.579329 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.582984 29152 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:36.583953 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.584908 29083 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:36.586598  1325 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.587730  1362 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:36.588450 29001 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2458) took 2320 ms (client timeout 2094 ms). Trace:
W20260709 13:12:36.588871 29001 rpcz_store.cc:269] 0709 13:12:34.267989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.813591 (+545602us) service_pool.cc:224] Handling call
0709 13:12:36.588237 (+1774646us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5400228654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:36.588377 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3982317440}
I20260709 13:12:36.591513 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:36.596845  1344 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:36.607919  1223 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:36.613117 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.614243 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.615047 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.615944 29151 raft_consensus.cc:2393] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:36.615890 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.616598 29157 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:36.616803 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.617343 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.617993 29157 raft_consensus.cc:2393] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 9: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:36.615243 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.618857 29153 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:36.622732 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:36.623441 29083 raft_consensus.cc:2368] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12.
I20260709 13:12:36.624148 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:36.625483 29083 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:36.629333 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.630582 29001 raft_consensus.cc:2368] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11.
I20260709 13:12:36.639043 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.639065 29004 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:36.639791 29079 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:36.640760 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:36.641279 29079 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:36.644665  1441 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:12:36.648409 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.649576 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.654208 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:36.618485  1328 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.656782 29152 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:36.655927 29159 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:36.658774 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.662149 29122 leader_election.cc:400] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11.
I20260709 13:12:36.662498 29122 leader_election.cc:403] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:36.660763 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.670794 29051 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:36.667673 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:36.673326 29712 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:36.660912  1193 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:12:36.674186  1193 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:12:36.674446 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.674513  1193 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:36.677301 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.677892 29153 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
W20260709 13:12:36.679356 29713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 570) took 4185 ms (client timeout 2573 ms). Trace:
W20260709 13:12:36.679785 29713 rpcz_store.cc:269] 0709 13:12:32.493986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.494124 (+   138us) service_pool.cc:224] Handling call
0709 13:12:36.679160 (+4185036us) spinlock_profiling.cc:243] Waited 3.3 s on lock 0x7b54006b0dd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:36.679264 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7582950656}
I20260709 13:12:36.681252 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:36.683213 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.684577 29006 raft_consensus.cc:2393] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:36.685704 29051 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:36.686640 29051 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:36.687489 29001 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:36.687634  1260 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:36.689841 29079 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:36.690567 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:36.690793 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.651794   934 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:36.675535  1193 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:36.692220  1260 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.693763 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.693840   912 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:36.694056 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.694425   912 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:36.695012   912 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:36.695226 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.695819 29008 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:36.696626 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:36.693778  1260 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:36.696875  1426 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.698971  1426 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.702607  1412 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.694461 29004 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:36.693851  1193 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:36.706429  1347 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:36.688601  1214 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:36.684284  1325 consensus_queue.cc:1059] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:12:36.657652  1349 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.708011 29154 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:36.716547 29157 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:36.749023 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.747221 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:36.823223 29076 raft_consensus.cc:2376] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14. Re-sending same reply.
I20260709 13:12:36.784483 29682 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:36.709957 29080 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:36.775589 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:36.741408 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.728885 29083 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:36.717000 29155 raft_consensus.cc:606] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:36.825845 29155 raft_consensus.cc:614] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:36.681105  1328 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:12:36.720090 29007 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:36.755585 29680 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:36.828985 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 73472.2us
I20260709 13:12:36.830344 29680 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:36.750571 29085 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:12:36.832794 29085 pending_rounds.cc:85] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 7
I20260709 13:12:36.833123 29085 pending_rounds.cc:107] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:12:36.833457 29085 raft_consensus.cc:2889] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:36.788973 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.697374 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.836261 29152 raft_consensus.cc:2410] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:12:36.780875 29153 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
W20260709 13:12:36.846839 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 62475.5us
I20260709 13:12:36.850878 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.885847 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.708350  1342 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:36.888736 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:36.891615 29079 raft_consensus.cc:2410] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:12:36.709811  1296 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:36.893422 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.809800  1223 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:36.888757  1412 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.747153   909 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:36.708962  1378 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:36.746842  1260 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.767858   988 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.845739  1362 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:36.894274 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.916659 29157 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:36.845589  1155 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:36.917265  1155 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.917589  1155 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:36.850891   897 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:36.924888 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:36.926101 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.747829  1304 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:36.929072  1263 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.795s	user 0.013s	sys 0.016s
W20260709 13:12:36.931552  1263 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.802s	user 0.013s	sys 0.017s
I20260709 13:12:36.932322  1155 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.934895  1155 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:36.913094 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 222563us
I20260709 13:12:36.915467 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:36.709407  1214 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:36.736456   934 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.936699  1362 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.895712   912 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.984790 29082 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:36.985476 29082 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:36.986445 29081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2204) took 1097 ms. Trace:
I20260709 13:12:36.898274  1223 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.954988 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:36.999221 29003 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:36.938253   988 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:36.956457  1448 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:12:37.003332   912 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:37.003995 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.004815 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.005487 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.006126 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.006769 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.007372 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:36.953518  1362 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:37.043247 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 153277us
I20260709 13:12:37.010592 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:37.025767 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 70866.2us
I20260709 13:12:37.044945 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2196) took 1158 ms. Trace:
I20260709 13:12:37.011512 29151 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.045401 29010 raft_consensus.cc:2393] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 14: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:37.045118 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:37.022562 29082 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.015770 29003 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:37.051260 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:37.052625 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.054421 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.060196 29002 raft_consensus.cc:2368] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:37.066406 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44428 (request call id 2191) took 1180 ms. Trace:
I20260709 13:12:37.067165 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.066912 29080 rpcz_store.cc:276] 0709 13:12:35.885830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.969363 (+ 83533us) service_pool.cc:224] Handling call
0709 13:12:37.066247 (+1096884us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b540034e254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:37.066355 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661345664}
I20260709 13:12:37.059506 29084 rpcz_store.cc:276] 0709 13:12:35.886980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.063211 (+176231us) service_pool.cc:224] Handling call
0709 13:12:36.240002 (+176791us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:36.241068 (+  1066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:36.241123 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:36.246531 (+  5408us) log.cc:844] Serialized 52 byte log entry
0709 13:12:36.517904 (+271373us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:36.526262 (+  8358us) log.cc:844] Serialized 14 byte log entry
0709 13:12:36.526665 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 13:12:36.526865 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.528580 (+  1715us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:36.660101 (+131521us) raft_consensus.cc:1685] finished
0709 13:12:36.660121 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.044789 (+384668us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b540027b5d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:37.044891 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12617,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":56,"raft.queue_time_us":798,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":7934,"spinlock_wait_cycles":374430336,"thread_start_us":211095,"threads_started":1,"wal-append.queue_time_us":270955}
I20260709 13:12:37.070936 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.071547 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:37.072260 29711 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:37.050956  1214 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.090384 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.099009 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:37.091846 29008 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:37.100016 29716 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:37.101611 29154 raft_consensus.cc:2368] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9.
I20260709 13:12:37.105924 29711 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:37.107758 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 62716.9us
W20260709 13:12:37.091575  1446 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 13:12:37.109439 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.051887  1456 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:37.112217 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.121773 29151 raft_consensus.cc:2410] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:12:37.129233 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.129979 29005 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:37.131292 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.134311 29050 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:37.135106 29050 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.090039   996 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 13:12:37.129175  1458 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:12:37.114387  1325 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:37.148797 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.151014 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52580 (request call id 29) took 2259 ms. Trace:
I20260709 13:12:37.153182 29155 rpcz_store.cc:276] 0709 13:12:34.888718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.987995 (+ 99277us) service_pool.cc:224] Handling call
0709 13:12:37.144670 (+2156675us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54003ed0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583b853 00007f411a80960a 00007f4113d8f7d4 00007f4113d8f762 00007f4113d8f691 00007f4113d8f60c 00007f4113d8e8c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:37.148334 (+  3664us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3973588736}
I20260709 13:12:37.139556  1342 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:37.155350 29154 raft_consensus.cc:2393] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 15: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:37.139909  1456 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:37.156793 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 97822us
W20260709 13:12:37.158296  1457 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:12:37.158308  1223 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:37.152872  1325 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.159165  1325 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:37.159507  1450 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:12:37.160493 29155 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:37.168637  1346 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:37.170912  1325 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.172077  1291 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:37.150975  1454 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:37.178418  1454 raft_consensus.cc:697] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:37.151154  1459 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:37.182089 29150 pending_rounds.cc:85] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 6
I20260709 13:12:37.182432 29150 pending_rounds.cc:107] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:12:37.182793 29150 raft_consensus.cc:2889] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:37.156520  1260 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:37.144022  1455 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:12:37.184316 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.184821 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:37.187922 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:37.138682 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 132480us
I20260709 13:12:37.192004 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:37.196357 29083 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:37.195695 29712 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
I20260709 13:12:37.201293 29005 raft_consensus.cc:2410] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:12:37.201932 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.205081 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:37.206136 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.179211   934 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:37.181600  1459 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.211402  1464 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:37.213299 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:37.214154 29157 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
W20260709 13:12:37.214936  1462 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:12:37.200270  1386 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.217700  1386 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.212055  1464 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.193823  1291 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.222486 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.219923 29080 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:37.219823  1386 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:37.224107  1454 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:36.986994 29081 rpcz_store.cc:276] 0709 13:12:35.889303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.245579 (+356276us) service_pool.cc:224] Handling call
0709 13:12:36.245868 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:36.246057 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:36.276195 (+ 30138us) log.cc:844] Serialized 14 byte log entry
0709 13:12:36.752104 (+475909us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b1800463300. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:12:36.950541 (+198437us) log.cc:844] Serialized 14 byte log entry
0709 13:12:36.984491 (+ 33950us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:36.984570 (+    79us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:36.984632 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.986176 (+  1544us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.986372 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30367,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":5266,"raft.queue_time_us":46236,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":5259,"spinlock_wait_cycles":29473536,"thread_start_us":197684,"threads_started":1,"wal-append.queue_time_us":769441}
I20260709 13:12:37.228363 29005 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 15: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:37.230861 29083 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:37.230892 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.234513 29682 leader_election.cc:400] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9.
I20260709 13:12:37.234984  1325 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:37.220160  1456 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.236284 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.210454   988 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:37.240187 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.216391  1459 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.238040 29682 leader_election.cc:403] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:12:37.244256 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.251842 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:37.255697  1464 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:37.256368  1464 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:37.258365  1378 raft_consensus.cc:3055] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:37.258733  1378 raft_consensus.cc:740] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:37.260581 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1738) took 1347 ms. Trace:
I20260709 13:12:37.261106 29716 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:37.260985 29077 rpcz_store.cc:276] 0709 13:12:35.912699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.294774 (+382075us) service_pool.cc:224] Handling call
0709 13:12:37.066418 (+771644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.066597 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.066619 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:37.066657 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:37.066690 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.066748 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.260426 (+193678us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b540034e250. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:37.260528 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773859712}
I20260709 13:12:37.261777  1344 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for earlier term 8. Current term is 9.
I20260709 13:12:37.235411  1461 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.269346  1461 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.236635  1466 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:12:37.230293  1291 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.274829 29082 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:37.275103  1291 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:37.275440 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:44428 (request call id 2209) took 1313 ms. Trace:
I20260709 13:12:37.278972 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.279547 29157 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:37.282740 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.283088 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.283795 29002 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:37.283751 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.284530 29001 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:37.285039 29002 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:12:37.285807 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.286369 29005 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:37.246625  1263 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:12:37.290494  1263 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:37.290843  1263 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:37.236481   934 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:12:37.293399   934 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:37.293748   934 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:37.294523   934 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.295701   934 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.275795 29082 rpcz_store.cc:276] 0709 13:12:35.962521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.354127 (+391606us) service_pool.cc:224] Handling call
0709 13:12:37.273654 (+919527us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400404a54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115838a3d 00007f411a808cd0 00007f4113d8d254 00007f4113d8d1e2 00007f4113d8d111 00007f4113d8d08c 00007f4113d8c342 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:37.275322 (+  1668us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426272128}
I20260709 13:12:37.298429 29001 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:37.299608 29050 leader_election.cc:304] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.299793 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:37.300916 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:37.301575 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:37.313755 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 63845.5us
I20260709 13:12:37.327122  1412 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:37.327561 29010 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:37.327823  1412 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:37.328496 29002 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:37.328744  1412 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.329527 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:37.330603 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.314790 29050 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:37.332180 29050 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:37.333012 29009 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:37.314111 29077 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:37.335119 29122 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:37.311921 29007 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:12:37.336324 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:37.330930 29006 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:37.339087 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.343627 29152 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:37.259532  1378 consensus_queue.cc:260] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:37.308194 32570 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.009s	sys 0.011s
I20260709 13:12:37.352238 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.347429 29157 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:37.291635  1263 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.312252  1395 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:37.366081 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.338734  1383 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:37.342176  1466 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:37.344066  1223 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:37.343930  1386 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:37.331909  1461 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:37.341434  1325 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:37.338568  1342 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:37.405306 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.408142 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.408869 29007 raft_consensus.cc:3060] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:37.412279 29153 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:37.411762 32630 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:37.415009 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.379424 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:37.341708  1326 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:37.397212 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.349964   988 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:12:37.419519 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.390566  1378 raft_consensus.cc:3060] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:37.392547  1223 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.385286  1455 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.390727  1466 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.435670 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.436445 29010 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:37.429081 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.429409 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:37.393409  1386 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.406013  1263 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.393205  1383 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:37.435504  1223 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.439378 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.479332 29004 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:37.444105 29157 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:37.457636 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.485224 29151 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:37.485556 29151 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:37.350152  1193 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:12:37.462093 29077 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:37.434948 28976 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.471571 29079 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:12:37.387209  1426 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:12:37.472353 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 220562us
I20260709 13:12:37.387475  1395 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.498417 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:37.421933  1325 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:37.485958 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 301052us
I20260709 13:12:37.494635 29682 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:37.454643  1466 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:37.472922  1467 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:12:37.473052  1386 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:37.431054  1362 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:37.502625 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.543776 28975 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:37.486614  1468 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:12:37.548300  1296 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:37.548787  1357 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.549269  1357 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:37.507467 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 72606us
I20260709 13:12:37.454842  1455 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.554548  1455 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:37.554603 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:37.555398 29159 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:37.510725 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.556357 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:37.556939 29157 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 11: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:37.557755 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:37.558348 29157 raft_consensus.cc:606] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:37.558696 29157 raft_consensus.cc:614] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:37.559993 29157 raft_consensus.cc:1240] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:37.510016  1346 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:37.500063  1464 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:12:37.572943 29005 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:37.521222  1472 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:12:37.524338  1461 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:37.525422  1479 raft_consensus.cc:2804] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:37.526976  1460 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:37.533326 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:37.536370  1383 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.540872  1325 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.430958  1326 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.545436 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.517517 29151 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.607787 29151 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:37.566109 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:37.567030  1454 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:37.570712  1357 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:37.576929 29713 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:37.577311 29153 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:37.571118  1214 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:37.609058 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 63682.4us
I20260709 13:12:37.611608 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:37.611228  1151 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:37.612125  1479 raft_consensus.cc:697] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:37.608606   909 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:37.610508 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:37.492419 29004 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:37.615290 29085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2214) took 1497 ms. Trace:
I20260709 13:12:37.491587 29007 raft_consensus.cc:2468] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
W20260709 13:12:37.573913  1328 consensus_meta.cc:287] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.728s	user 0.006s	sys 0.005s
I20260709 13:12:37.620980 29157 raft_consensus.cc:606] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:37.638914 29157 raft_consensus.cc:614] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:37.620975 29125 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:37.607218  1342 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.637733 29151 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:37.640779 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:37.627466 29085 rpcz_store.cc:276] 0709 13:12:36.118301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.845177 (+726876us) service_pool.cc:224] Handling call
0709 13:12:36.845527 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:37.056366 (+210839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.056409 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:37.056913 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:12:37.183087 (+126174us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:37.395201 (+212114us) log.cc:844] Serialized 14 byte log entry
0709 13:12:37.614997 (+219796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.615073 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:37.615110 (+    37us) raft_consensus.cc:1685] finished
0709 13:12:37.615121 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.615239 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31910,"prepare.queue_time_us":112516,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":105310,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":13790,"thread_start_us":668179,"threads_started":4,"wal-append.queue_time_us":129393}
I20260709 13:12:37.610545  1460 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:37.650944 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:37.656637 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:37.657709 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:37.658866 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:37.645931 29151 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:12:37.663040 29151 pending_rounds.cc:85] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 8
I20260709 13:12:37.664434 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:37.664839 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:37.665180 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:37.660491 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:37.672135 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 104391us
I20260709 13:12:37.672590 29711 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.626899   909 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:37.619369  1346 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:12:37.643953  1328 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11.
I20260709 13:12:37.673755 29151 pending_rounds.cc:107] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:12:37.674017 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:37.675005 29708 raft_consensus.cc:1240] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:37.672246 29712 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:37.667726 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 116544us
I20260709 13:12:37.676409 29153 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:12:37.648366  1461 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:37.682106 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.662683  1244 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:37.695488 29084 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
I20260709 13:12:37.684234 29711 raft_consensus.cc:2468] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:37.617450  1454 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.685521 29151 raft_consensus.cc:2889] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:37.682816 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 334941us
I20260709 13:12:37.686326 32570 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:37.686571 29125 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:37.707075 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:37.708413 29125 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:37.670742  1193 raft_consensus.cc:2764] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:37.642796  1459 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.627142  1214 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.670619  1123 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:37.714954 28902 catalog_manager.cc:5697] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 9, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 9 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:37.721150 29156 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:37.711095 32570 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:12:37.673448  1376 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.678112   988 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:37.762163   988 raft_consensus.cc:697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:37.763206   988 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.650353  1342 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:37.764932  1469 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:12:37.766685  1155 consensus_queue.cc:1059] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:12:37.768091  1480 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:12:37.768970  1123 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:37.769695  1475 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:12:37.769888  1123 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.770913 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1925) took 1006 ms. Trace:
I20260709 13:12:37.771303 29150 rpcz_store.cc:276] 0709 13:12:36.764971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.766651 (+  1680us) service_pool.cc:224] Handling call
0709 13:12:37.181883 (+415232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:37.193877 (+ 11994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.193912 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:37.194359 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:12:37.312424 (+118065us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:37.312456 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.312518 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:37.770705 (+458187us) raft_consensus.cc:1685] finished
0709 13:12:37.770727 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.770855 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":92079,"threads_started":1,"wal-append.queue_time_us":213089}
I20260709 13:12:37.713271  1460 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.775395  1460 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:37.752161  1487 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:37.691797  1151 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:37.757596  1483 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:37.761314  1244 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:37.766955 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2211) took 1763 ms. Trace:
I20260709 13:12:37.797860 29076 rpcz_store.cc:276] 0709 13:12:36.003835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.824076 (+820241us) service_pool.cc:224] Handling call
0709 13:12:36.824399 (+   323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:37.065358 (+240959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.065386 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:37.071065 (+  5679us) log.cc:844] Serialized 52 byte log entry
0709 13:12:37.337288 (+266223us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:37.348928 (+ 11640us) log.cc:844] Serialized 14 byte log entry
0709 13:12:37.349116 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.349186 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:37.766694 (+417508us) raft_consensus.cc:1685] finished
0709 13:12:37.766720 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.766886 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":695,"prepare.queue_time_us":13416,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":3917,"raft.queue_time_us":114056,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":26217,"thread_start_us":446594,"threads_started":2,"wal-append.queue_time_us":369741}
W20260709 13:12:37.781385 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.710171  1296 raft_consensus.cc:697] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:37.801213  1296 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:37.792514 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:37.794915 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 119177us
W20260709 13:12:37.806731 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 233861us
W20260709 13:12:37.808231 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:37.811437 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.814697 28902 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 8, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 8 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:37.816983  1464 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:37.817471  1464 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:37.818423  1464 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.822332 29680 leader_election.cc:304] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:37.823170  1455 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:37.823473 29680 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:37.826525  1155 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:37.833236 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:37.833720 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32836 (request call id 714) took 1025 ms. Trace:
I20260709 13:12:37.833847 29007 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:37.834131 29156 rpcz_store.cc:276] 0709 13:12:36.808521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.915136 (+106615us) service_pool.cc:224] Handling call
0709 13:12:37.833550 (+918414us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54000942d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:37.833667 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1766387072}
I20260709 13:12:37.836166 29051 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:37.841215 29007 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:37.846544 28902 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:37.823707  1455 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.715477  1461 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:37.814168 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 105830us
I20260709 13:12:37.851694 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:37.843935 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 138812us
W20260709 13:12:37.756404  1458 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:12:37.855976 29680 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:37.751834  1328 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:37.819878  1464 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:37.858809  1481 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:12:37.858928   996 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:37.859735 29713 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:37.859817 28902 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 7 to 10, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 10 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:37.889408 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1938) took 1120 ms. Trace:
I20260709 13:12:37.862573 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.889796 29155 rpcz_store.cc:276] 0709 13:12:36.768607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.162117 (+393510us) service_pool.cc:224] Handling call
0709 13:12:37.162375 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.162579 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:37.394623 (+232044us) log.cc:844] Serialized 14 byte log entry
0709 13:12:37.889104 (+494481us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:37.889150 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:37.889191 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.889254 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.889356 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14629,"prepare.queue_time_us":94442,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":28717,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":4844,"thread_start_us":710297,"threads_started":3}
I20260709 13:12:37.893157 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:37.858566 29711 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:37.870972  1502 raft_consensus.cc:2804] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:37.875912 29158 raft_consensus.cc:2410] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:37.881846 29152 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
W20260709 13:12:37.888116 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 206148us
I20260709 13:12:37.896874 29154 raft_consensus.cc:1275] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:37.882920 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:37.897854 29157 raft_consensus.cc:606] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:37.797485  1483 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:37.899955 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.905567  1502 raft_consensus.cc:697] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:37.906154 28905 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52674 (request call id 261) took 18690 ms (client timeout 15000 ms). Trace:
I20260709 13:12:37.906488  1502 consensus_queue.cc:237] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.855808  1151 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:37.858413  1465 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:37.857848  1362 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:37.913762 29007 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:12:37.916209 28902 catalog_manager.cc:5697] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 8, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 8 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:37.918704 29004 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:37.908281  1491 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:37.888923   996 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:37.919567  1455 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:37.903271  1193 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:37.903297  1118 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:37.906522 28905 rpcz_store.cc:269] 0709 13:12:19.216095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:19.223816 (+  7721us) service_pool.cc:224] Handling call
0709 13:12:37.905936 (+18682120us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:28.158836 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:28.227843 (+ 69007us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:28.229330 (+  1487us) write_op.cc:435] Acquired schema lock
0709 13:12:28.235631 (+  6301us) tablet.cc:662] Decoding operations
0709 13:12:28.595171 (+359540us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:28.595253 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:28.595277 (+    24us) tablet.cc:685] Acquiring locks for 29 operations
0709 13:12:28.597549 (+  2272us) tablet.cc:701] Row locks acquired
0709 13:12:28.597573 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:12:28.597728 (+   155us) write_op.cc:270] Start()
0709 13:12:28.597842 (+   114us) write_op.cc:276] Timestamp: P: 1783602748597676 usec, L: 0
0709 13:12:28.597863 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:12:28.690523 (+ 92660us) log.cc:844] Serialized 11331 byte log entry
0709 13:12:28.863076 (+172553us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b54000f7dd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411583ce28 00007f4119277581 00007f4119276d58 00007f411927b9f1 00007f411927b9a9 00007f411927b939 00007f411927b901 00007f411927ab2d 000000000038f354 000000000038f199
0709 13:12:29.191840 (+328764us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:29.751990 (+560150us) op_driver.cc:464] REPLICATION: finished
0709 13:12:29.986630 (+234640us) write_op.cc:301] APPLY: starting
0709 13:12:29.986764 (+   134us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:30.310234 (+323470us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b18001099c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41155800ec 00007f4115582af7 00007f4115583fb0 00007f41192981bb 00007f4119298532 00007f41192dfb93 00007f411918cde3 00007f411918c6ab 00007f411918de77
0709 13:12:30.754053 (+443819us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:30.754110 (+    57us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:33.391550 (+2637440us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:33.391966 (+   416us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 13:12:33.392011 (+    45us) write_op.cc:312] APPLY: finished
0709 13:12:33.392643 (+   632us) log.cc:844] Serialized 252 byte log entry
0709 13:12:33.578309 (+185666us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:33.661834 (+ 83525us) write_op.cc:454] Released schema lock
0709 13:12:33.661943 (+   109us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:33.728600 (+ 66657us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:33.731039 (+  2439us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22027,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":220895,"apply.run_cpu_time_us":71662,"apply.run_wall_time_us":3715962,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125347,"num_ops":29,"prepare.queue_time_us":572026,"prepare.run_cpu_time_us":14387,"prepare.run_wall_time_us":713408,"raft.queue_time_us":385046,"raft.run_cpu_time_us":34923,"raft.run_wall_time_us":417888,"replication_time_us":1015527,"spinlock_wait_cycles":387203072,"thread_start_us":1241301,"threads_started":5,"wal-append.queue_time_us":372134,"wal-append.run_cpu_time_us":54083,"wal-append.run_wall_time_us":2871763}]]}
I20260709 13:12:37.926116  1304 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.926609  1244 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.908761  1500 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:37.903010  1357 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:37.928342  1498 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:37.930311  1386 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.934319  1492 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:37.750270  1479 consensus_queue.cc:237] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.855361   909 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.920138 29712 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:37.938428 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:37.807756  1376 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:37.908643  1395 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:37.993641 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 55298.4us
I20260709 13:12:37.971457  1487 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:37.978245  1326 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:37.989202  1484 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:12:37.910116  1483 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.911327  1465 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:37.986111 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:37.979341 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 79447.8us
I20260709 13:12:37.919970   897 raft_consensus.cc:2804] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:37.992219 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 133475us
I20260709 13:12:37.970861   996 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:37.983304 28902 catalog_manager.cc:5697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:37.992901  1263 consensus_meta.cc:287] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.002s
W20260709 13:12:38.000010  1482 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:12:38.004515  1510 raft_consensus.cc:2764] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:12:38.004678  1376 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:38.005817  1346 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:38.006769 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.007417  1386 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.008186  1151 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.013872 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:38.051298 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:37.992347  1478 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:12:37.983846  1500 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:38.030073 29125 leader_election.cc:400] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12.
I20260709 13:12:37.985899  1118 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:37.996912  1493 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:38.053334 29125 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:38.031948  1487 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.036489 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2608) took 1474 ms. Trace:
I20260709 13:12:38.029079  1492 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:38.056434 29158 rpcz_store.cc:276] 0709 13:12:36.562155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.562287 (+   132us) service_pool.cc:224] Handling call
0709 13:12:38.036289 (+1474002us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400398d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:38.036414 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3004246272}
I20260709 13:12:38.038493 29001 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:12:38.044266 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 248462us
I20260709 13:12:37.989882  1304 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:38.046757  1223 consensus_meta.cc:287] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
I20260709 13:12:38.046494 28902 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 9, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 9 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:38.035414  1345 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.060390  1260 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:38.060453 29051 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:38.061196  1386 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:38.061393 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:38.062387  1508 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:12:38.062861  1495 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:38.072932 29682 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:38.018576  1466 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.081223 32570 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:38.083115 29712 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:38.095350 32570 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:38.095767 32570 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:38.096097 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.086695 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.096999 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:38.098121  1503 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:38.098685  1503 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:38.099160  1503 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:38.057689  1500 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.080463 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:38.105309 29125 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:38.080691  1345 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.088451 28902 catalog_manager.cc:5697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 14. New cstate: current_term: 14 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:38.058825  1296 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:38.117156  1459 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.120868  1492 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.084738  1223 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.126195  1223 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:38.126899 29159 raft_consensus.cc:2393] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:38.045885  1513 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:12:38.050271  1346 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:38.132769 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 46142.6us
I20260709 13:12:38.133945 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:38.141206 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.141989 29078 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:38.143782 29077 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:38.149887 29712 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:12:38.151167  1459 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:12:38.115795  1511 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:38.081558  1326 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:12:38.199292 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46895us
I20260709 13:12:38.153823 29716 raft_consensus.cc:2368] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12.
I20260709 13:12:38.149601 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:38.204569 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55025.7us
I20260709 13:12:38.205324 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.206388 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.154289 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:38.193419 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.089195  1455 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:38.092650  1493 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:38.192785 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 142102us
I20260709 13:12:38.194451 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:38.209678 29076 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:38.153744 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:38.195217 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 43484.1us
I20260709 13:12:38.213376 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:38.214412 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:38.215591   988 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:38.216470   988 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:38.216787   988 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:38.217217   988 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:38.217170  1493 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.218931  1493 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.199471 29081 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:38.226452 29076 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:38.221115  1493 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.215657 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:38.227243   988 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.232203  1328 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.237984 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.239609 29077 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:38.244755 29081 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:38.241405 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:38.131827  1513 consensus_queue.cc:1243] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer ca45420e033847489ad989c1c6eed3ef log is divergent from this leader: its last log entry 11.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:12:38.242486 32570 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.247911 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:38.250434 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.154603  1223 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:38.240891 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:38.154249  1460 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:38.135264  1490 raft_consensus.cc:2764] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:12:38.255463 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:38.261068 29081 raft_consensus.cc:2393] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:38.257027 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.154443  1123 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:38.144860   897 raft_consensus.cc:697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:38.194574  1491 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:38.254000 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.240836  1412 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:38.263783 28902 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:38.243242  1504 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:38.248863  1511 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:38.277280 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.259902  1426 raft_consensus.cc:3055] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:38.285631 28902 catalog_manager.cc:5697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 5, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 5 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:38.266672  1491 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.085168  1465 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.262575  1325 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:38.295274 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:38.295761 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:38.296314 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.289683 29077 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:38.278365 32570 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:38.275835 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.292768 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.288247 29079 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:38.084414  1510 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:12:38.296326 29083 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:38.298091 29081 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:38.215032  1499 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:38.276526  1123 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:38.266525  1460 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.326054 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.323500 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:38.301685 29084 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:38.285983  1426 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:38.298041 28902 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 9, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:38.333458 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 91244.5us
I20260709 13:12:38.304980 29717 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:38.336258 32570 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:38.291733  1491 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:38.348212 28902 catalog_manager.cc:5697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:38.292636  1304 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.291122  1455 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:38.351522 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:38.356757 29084 raft_consensus.cc:2368] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13.
I20260709 13:12:38.356850 29083 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
W20260709 13:12:38.347738  1512 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 13:12:38.359551 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:38.360476 28976 connection.cc:256] RPC call timeout handler was delayed by -1.3537743050000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.36040000000000005s for a response
I20260709 13:12:38.334957  1325 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:38.361881 29083 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:38.357738  1515 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:12:38.334617  1511 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.351166   897 consensus_queue.cc:237] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.380143   988 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:38.381249 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:38.382472 29084 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:12:38.375016  1346 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.391402 29083 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:38.399230 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:38.351233  1518 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:12:38.409516   996 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.411808   996 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:38.397454  1378 consensus_meta.cc:287] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.960s	user 0.011s	sys 0.000s
I20260709 13:12:38.416848 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:38.417457 29083 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:38.421192 29082 raft_consensus.cc:2468] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:38.424955 29083 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:38.366992  1325 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.401997  1345 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.396965  1513 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:12:38.403075  1508 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:38.405565  1466 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:38.421195  1525 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:12:38.393586  1426 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.433236  1455 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:38.432135 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 225789us
I20260709 13:12:38.441574 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:38.453786  1499 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:38.455586 29125 leader_election.cc:400] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13.
I20260709 13:12:38.455405  1499 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:38.456022 29125 leader_election.cc:403] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:12:38.456831 29125 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:38.457038  1499 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:38.458384 29048 leader_election.cc:400] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:38.458678 29048 leader_election.cc:403] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:12:38.461823  1500 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.464228  1376 raft_consensus.cc:2804] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:38.438407  1483 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.464833  1376 raft_consensus.cc:697] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:38.465806  1376 consensus_queue.cc:237] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.466318  1483 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:38.466938  1499 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13.
W20260709 13:12:38.473767  1531 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:12:38.439145  1342 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.468904 29125 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:38.477655 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:38.477994 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 95776.9us
I20260709 13:12:38.481021 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.481019 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.482576 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:38.483644 29083 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:38.485885 29048 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.443545  1508 raft_consensus.cc:697] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:38.458487  1464 consensus_meta.cc:287] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.005s
I20260709 13:12:38.486747  1223 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:38.446563   996 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:12:38.488821  1529 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:38.489107  1535 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:38.497555  1491 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.498243  1503 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.483330  1342 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:38.483207  1456 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:38.500623  1454 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.419031  1123 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.489876 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.504230 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:38.515928 29083 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:38.539590 29048 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.474788  1483 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 13:12:38.542841 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61919.2us
I20260709 13:12:38.495851   996 consensus_queue.cc:1243] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:12:38.548194  1539 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:38.548851  1488 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.499962  1426 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.495518  1223 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:38.551131  1523 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:38.537988  1527 log.cc:927] Time spent T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:12:38.554281 29713 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:38.517505  1456 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:38.547742  1328 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:38.517238  1460 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.565333 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.565929 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.549757  1342 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:38.550789  1500 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:38.564397  1454 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:38.596690 29010 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:38.574759  1539 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:38.583482 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:38.599601  1378 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.600826 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.602080 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.602880  1502 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:38.603755  1502 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:38.576632  1523 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:38.605598 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:38.606161 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:38.606940  1536 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:12:38.614521 29005 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:38.615475 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:38.616696 29009 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:38.620504  1378 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:38.620812  1522 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:38.631873 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2620) took 1257 ms. Trace:
I20260709 13:12:38.632402 29001 rpcz_store.cc:276] 0709 13:12:37.374322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.377293 (+  2971us) service_pool.cc:224] Handling call
0709 13:12:38.631619 (+1254326us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b5400228654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:38.631804 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418112768}
I20260709 13:12:38.633946 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1942) took 1294 ms. Trace:
I20260709 13:12:38.634351 29159 rpcz_store.cc:276] 0709 13:12:37.339938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.556120 (+216182us) service_pool.cc:224] Handling call
0709 13:12:38.633683 (+1077563us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54002b5cd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:38.633873 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310390144}
W20260709 13:12:38.635751  1541 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:12:38.615592 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:38.640779 29004 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:38.640859 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:38.585583  1491 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:38.591140 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.548779  1508 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.641404 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:38.652935 29006 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:38.641490 29009 raft_consensus.cc:3060] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:38.653017 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:38.642177 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:38.651101 29010 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:38.654356 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:38.657492 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 65010.3us
I20260709 13:12:38.658470 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:38.603631 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.662590 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.662719 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:38.588932  1529 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:38.596570  1494 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:12:38.666074 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 62519.9us
I20260709 13:12:38.664163  1345 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:38.669756  1378 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:38.670212  1223 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.672492 29048 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.672981 28976 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:38.676882  1546 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:12:38.676716 29122 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:38.639043  1539 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.691031 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:38.689074 29002 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:38.577216  1488 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.638680  1454 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.674060  1345 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:38.663697  1538 raft_consensus.cc:2764] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
W20260709 13:12:38.686573  1461 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.822s	user 0.002s	sys 0.008s
I20260709 13:12:38.665720  1491 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:38.672091  1494 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:38.694535 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:38.700771 29009 raft_consensus.cc:2468] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:38.663419  1464 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 13:12:38.705974 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.694895  1529 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:38.717278  1488 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.696695  1325 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.702411  1345 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:38.704555  1304 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:38.705794  1378 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:38.733608 29006 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:38.734012  1454 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.733846  1378 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.711355  1494 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:38.727419 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:38.729681  1263 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:38.728538  1538 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:12:38.711758  1500 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:12:38.709880  1502 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:38.704396  1328 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:38.710816  1461 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:38.735843  1378 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.746578 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:38.737668  1464 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.750329 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:38.750900 29156 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:38.742167  1304 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:38.752359 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:38.758946 29710 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
W20260709 13:12:38.762632 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.765098 29150 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:12:38.766503 29157 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:38.768311 29156 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:38.769338 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:38.769578 29151 raft_consensus.cc:1275] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:12:38.776513 29158 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:38.744199  1491 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:12:38.780865  1214 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.784536 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:38.766580  1304 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:38.744246  1502 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.772375  1328 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.756973  1464 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:38.814674  1304 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.775022  1345 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.829393  1502 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.830317  1357 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:38.822057  1538 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.840934 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:38.819000  1328 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.834475  1357 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:38.853875 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:38.857199 29158 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:38.859848 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:38.860602 29158 raft_consensus.cc:2410] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:12:38.864187 29159 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:38.865047 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:38.865483 29159 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:38.868629 29153 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:38.868486 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:38.870443 29156 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:38.870636 29159 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:38.871683 29150 raft_consensus.cc:2393] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 14: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:38.872370 29151 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:38.874831 29155 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:38.878165 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.889772 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:38.890746 29712 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:38.893226  1493 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:38.842003  1538 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:38.899923 29009 raft_consensus.cc:1240] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:38.898451 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:38.904539 29153 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:38.907348 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:38.909515 29151 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.918713 29712 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:38.923899 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:38.926767 29009 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:38.925922 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:38.927677 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:38.928609 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:38.926805 29048 leader_election.cc:336] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.698s (SENT)
I20260709 13:12:38.931496 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:38.932272 29714 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:38.899196  1263 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:12:38.934294  1263 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:38.935946  1263 consensus_queue.cc:1059] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:12:38.923743  1550 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:12:38.941016 29151 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:38.944923 29150 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:38.945654 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:38.937498  1553 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:12:38.950629 29712 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:38.952500 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:38.953207 32630 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:38.954213 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 213541us
I20260709 13:12:38.960255 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:38.961259 32630 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:38.957240 29714 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:38.961750 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:38.973608 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:38.974363 32630 raft_consensus.cc:2764] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:12:38.994328 32630 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:38.962481  1426 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:38.983527  1123 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:38.969342  1500 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:38.989780  1493 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:12:38.994352  1555 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:12:38.941911  1539 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.022332  1539 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.986493  1464 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:39.023409  1118 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:39.029166  1465 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:39.030323  1538 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:38.986903  1499 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:12:39.007061  1345 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:39.019785  1493 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:39.074198  1514 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.958s	user 0.004s	sys 0.000s
W20260709 13:12:39.074661  1558 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:12:39.107446 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 226432us
W20260709 13:12:39.114414 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.115993 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 155714us
I20260709 13:12:39.117607 28976 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:39.130894 29008 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:39.140180 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:39.172808 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.174465 29010 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:39.177052 29153 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:12:39.177496 29153 pending_rounds.cc:85] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 7
I20260709 13:12:39.177825 29153 pending_rounds.cc:107] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:12:39.178179 29153 raft_consensus.cc:2889] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:39.199494 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.203685 29125 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:39.208534 29716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47384 (request call id 899) took 1548 ms. Trace:
I20260709 13:12:39.209076 29716 rpcz_store.cc:276] 0709 13:12:37.659785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.661174 (+  1389us) service_pool.cc:224] Handling call
0709 13:12:39.207997 (+1546823us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b5400650ed4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:39.208301 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087824640}
I20260709 13:12:39.213444 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 585) took 1067 ms. Trace:
I20260709 13:12:39.213943 29708 rpcz_store.cc:276] 0709 13:12:38.145597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.145762 (+   165us) service_pool.cc:224] Handling call
0709 13:12:39.212988 (+1067226us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54000154d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:39.213266 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":797960448}
I20260709 13:12:39.216127 29710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 586) took 1070 ms. Trace:
I20260709 13:12:39.216610 29710 rpcz_store.cc:276] 0709 13:12:38.145945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.146091 (+   146us) service_pool.cc:224] Handling call
0709 13:12:39.215665 (+1069574us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54006b0dd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:39.215904 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133196160}
I20260709 13:12:39.222471 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 584) took 1077 ms. Trace:
I20260709 13:12:39.222951 29713 rpcz_store.cc:276] 0709 13:12:38.145248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.146130 (+   882us) service_pool.cc:224] Handling call
0709 13:12:39.222078 (+1075948us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b5400650254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:39.222291 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577889152}
I20260709 13:12:39.229585 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:39.234305 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:39.235625 29082 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:39.244597 29079 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:39.257409  1118 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:39.259922  1118 consensus_queue.cc:1059] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:39.261363  1118 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:39.264562  1118 raft_consensus.cc:2804] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:39.265271  1118 raft_consensus.cc:697] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:39.267961  1123 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:39.268229   909 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:39.290200  1328 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:39.283180  1123 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:39.331122 29157 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:39.339489  1118 consensus_queue.cc:237] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.340092  1559 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:12:39.341566 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.344118 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.344348  1493 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.345461 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.347180  1493 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:39.347687 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.348754 29158 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:39.349608 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:39.351181 29158 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:39.351504 29158 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:39.352030 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.352468 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.356155  1562 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.356886 29152 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:39.357807  1539 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:39.358325  1539 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:39.358790  1539 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:39.288597  1494 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:39.360244 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 144913us
I20260709 13:12:39.362178 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:39.362728  1494 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:39.364192  1567 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:39.365433  1328 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:39.366734  1552 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:39.367175 29005 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
W20260709 13:12:39.368729 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52973.4us
I20260709 13:12:39.369566 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:39.348528  1538 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.372004  1538 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.373610  1522 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:39.367197  1552 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.376864  1552 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:39.377820  1535 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:39.377943 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2241) took 1089 ms. Trace:
I20260709 13:12:39.378531 29078 rpcz_store.cc:276] 0709 13:12:38.288002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.288162 (+   160us) service_pool.cc:224] Handling call
0709 13:12:38.288428 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:38.288618 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:38.354900 (+ 66282us) log.cc:844] Serialized 14 byte log entry
0709 13:12:39.159251 (+804351us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b1800215920. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:12:39.377198 (+217947us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:39.377312 (+   114us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:39.377431 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.377562 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.377736 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49900,"prepare.queue_time_us":44593,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":50229,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":212,"spinlock_wait_cycles":17531136,"thread_start_us":213047,"threads_started":2,"wal-append.queue_time_us":997318}
W20260709 13:12:39.380688 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 240888us
W20260709 13:12:39.386409 29048 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.725s (SENT)
I20260709 13:12:39.386976 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:39.343058 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.346763 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:39.375692  1538 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:39.402899 29153 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:39.417361 29159 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:39.357033  1562 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:39.433045 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 99698.3us
W20260709 13:12:39.439280 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 255665us
I20260709 13:12:39.441256 29158 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:39.448076 29158 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:39.456790  1502 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:39.458073  1466 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.458977  1466 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:39.461333  1523 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:39.461766  1466 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:39.462489 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 67209.1us
I20260709 13:12:39.466460 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:39.468463  1455 consensus_queue.cc:1048] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:39.470045  1455 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:39.472237  1123 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.473706  1123 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:39.475664 29158 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:12:39.476794 29158 pending_rounds.cc:85] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 10
I20260709 13:12:39.477936 29158 pending_rounds.cc:107] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:12:39.478394 29158 raft_consensus.cc:2889] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:39.455823 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:39.483779 29155 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 11. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:12:39.482718 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:39.477319  1562 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:39.500707 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:39.500999  1503 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:39.502942  1521 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 1.213s	user 0.001s	sys 0.000s
I20260709 13:12:39.510007  1291 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:39.510545  1291 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:39.518507 29711 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:39.518831  1500 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:39.519289  1500 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:39.519851 29050 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:39.520506  1500 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.521592 29050 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:39.522516  1500 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:39.522424 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:39.524149 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:12:39.535503  1357 consensus_meta.cc:287] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.002s
W20260709 13:12:39.536911  1549 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.944s	user 0.000s	sys 0.002s
I20260709 13:12:39.511898  1291 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:39.545966 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 174280us
I20260709 13:12:39.487967  1118 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:39.552744  1151 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
W20260709 13:12:39.553223 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 133434us
I20260709 13:12:39.553448  1263 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:39.553687  1507 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:12:39.555110 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 120196us
I20260709 13:12:39.555809  1526 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:39.556522  1509 raft_consensus.cc:3055] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:12:39.574038 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 43520.5us
W20260709 13:12:39.557586  1519 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.478s	user 0.003s	sys 0.001s
I20260709 13:12:39.491987  1466 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:12:39.573969  1509 raft_consensus.cc:740] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:39.581619  1502 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:39.582463  1509 consensus_queue.cc:260] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.583160  1511 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:39.586202  1456 consensus_queue.cc:1048] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:12:39.586789 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.586987 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:39.587839 29078 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:39.588893  1386 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:39.589455  1386 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:39.590505  1386 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.593194 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:39.595551 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:39.597186  1490 raft_consensus.cc:2764] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:12:39.601042  1362 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:39.602988 29006 raft_consensus.cc:1275] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
W20260709 13:12:39.604715 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:39.569442  1328 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.605646  1509 raft_consensus.cc:3060] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:39.605927  1328 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:39.605120 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.576808  1357 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.573791  1526 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.636363 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.646626 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:39.652412 29158 raft_consensus.cc:2393] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 11: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:39.631975  1526 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:39.654331 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.616145  1539 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.665270 29078 raft_consensus.cc:2468] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:39.613484  1118 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:39.659232 29005 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:39.637986  1456 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.627262  1552 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.671789 29682 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:39.674713   996 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.675251  1346 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:39.676503 29150 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:39.676339 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.667797  1460 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:39.666452  1291 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:39.676339  1518 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.695700  1376 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:12:39.698041  1395 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:39.699194 29084 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:12:39.655434  1567 consensus_queue.cc:1059] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:12:39.698084  1155 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:39.705013 29156 raft_consensus.cc:2410] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:12:39.709527 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.711529 29079 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:39.714270 29680 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:39.700923  1460 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:39.715039   996 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:39.720269 29002 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
W20260709 13:12:39.720548 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 48878.5us
I20260709 13:12:39.720602  1214 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:12:39.723240  1456 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:39.724426 29051 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:39.724980  1518 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.725863 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:39.727377  1328 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:39.729022  1357 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:39.733260  1567 consensus_queue.cc:1243] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:12:39.698169  1576 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:12:39.736865 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:39.752818 29078 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:39.753253  1539 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:39.745077 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.753688  1518 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:39.753777 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44833.5us
I20260709 13:12:39.703547  1395 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:39.738405 29004 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:39.751636 29085 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:12:39.748021 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 82846.7us
I20260709 13:12:39.777204 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:39.777884 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:39.778779 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 156277us
W20260709 13:12:39.779408  1465 consensus_meta.cc:287] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.011s
I20260709 13:12:39.747275  1466 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:39.783712 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.803483 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.804262 29009 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:39.750896  1552 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:39.758469  1214 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:39.784981  1460 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.786073  1526 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.749540   996 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:39.788334  1594 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.055s	user 0.004s	sys 0.000s
I20260709 13:12:39.796675 29084 raft_consensus.cc:1275] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:12:39.796725 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 218156us
I20260709 13:12:39.794126 29004 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:39.783607 29050 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:39.799090 29158 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
W20260709 13:12:39.799430  1304 consensus_meta.cc:287] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.964s	user 0.010s	sys 0.010s
I20260709 13:12:39.813278 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.800122  1576 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:39.751104  1456 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:39.788208  1492 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:12:39.823410 29009 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:39.821949 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.797524  1395 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:39.848606 29079 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 10.
I20260709 13:12:39.847714 29008 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
W20260709 13:12:39.849363 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 123647us
I20260709 13:12:39.854208 29714 raft_consensus.cc:1275] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:39.855067 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:39.856005 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 72517us
I20260709 13:12:39.858291 29010 raft_consensus.cc:2393] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 14: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:39.833284  1576 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:39.868299 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:39.868736  1576 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:39.869560  1214 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:39.869637 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:39.870213  1214 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:39.870443 29008 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:39.870762  1214 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:39.871774   897 consensus_queue.cc:1048] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:12:39.872505 29713 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:39.873528  1522 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:39.874993 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:39.876747  1244 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:39.877285  1244 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:39.878126 29125 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:39.879652  1244 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:39.881904 28975 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:39.882634 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.884590  1155 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:39.884944  1582 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:12:39.857755  1501 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.266s	user 0.015s	sys 0.018s
I20260709 13:12:39.888624 29006 raft_consensus.cc:2410] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:12:39.889797 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.866683 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.858184  1362 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.870332  1576 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:39.860623  1591 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:39.805174  1535 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.113s
I20260709 13:12:39.968115 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:39.958755  1501 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.555s	user 0.016s	sys 0.019s
I20260709 13:12:39.973692 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:39.932929  1518 raft_consensus.cc:993] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:12:39.977694 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:39.976585 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.853178  1304 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:39.972215 29048 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.143s (SENT)
I20260709 13:12:39.988744 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:39.933360  1582 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:39.989293 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 107440us
I20260709 13:12:39.986335 29003 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:39.965221  1591 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.995982 29005 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:40.002830 29008 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:39.983791  1500 raft_consensus.cc:2804] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:40.014735 29157 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:39.962170  1503 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:12:40.020324 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:40.023705 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:40.012102  1346 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.027212 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:39.970613  1362 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:39.998849  1591 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:40.014320 29004 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.033505 29004 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:40.034801 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.035748 29005 raft_consensus.cc:2468] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
W20260709 13:12:40.040495 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.016258  1509 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:40.017581  1507 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:40.051512 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.053728 32630 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.141s
I20260709 13:12:40.047614   912 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.056068 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:40.048198  1509 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:40.062950 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:40.084545 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 112694us
I20260709 13:12:40.030738  1346 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.101359 29157 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:40.105304 29001 raft_consensus.cc:1240] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:40.076865  1526 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:40.116672 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.064342  1509 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:40.105077 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2512) took 2067 ms. Trace:
W20260709 13:12:40.149721 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44958.7us
I20260709 13:12:40.153275 29158 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:40.132823 29003 raft_consensus.cc:2376] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14. Re-sending same reply.
W20260709 13:12:40.183125 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 141825us
I20260709 13:12:40.188269 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:40.193430 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.194233 29003 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:40.093240  1501 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:40.202621 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:40.057941  1582 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.133543 29007 rpcz_store.cc:276] 0709 13:12:37.916968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.917073 (+   105us) service_pool.cc:224] Handling call
0709 13:12:37.917373 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:37.971783 (+ 54410us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:37.993518 (+ 21735us) log.cc:844] Serialized 14 byte log entry
0709 13:12:38.628189 (+634671us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b18003ae700. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a
0709 13:12:38.628419 (+   230us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:38.628908 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:12:38.629036 (+   128us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:38.629070 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.629139 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:39.556535 (+927396us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:39.965653 (+409118us) raft_consensus.cc:1685] finished
0709 13:12:39.965671 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.984264 (+ 18593us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72245,"prepare.queue_time_us":16690,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":85189,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":32967,"spinlock_wait_cycles":18142976,"thread_start_us":22684,"threads_started":1,"wal-append.queue_time_us":30983}
I20260709 13:12:40.217338  1582 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:40.217443 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.218961 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:40.219632 29007 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:40.219918  1501 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.226596 29004 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:12:40.223909 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:40.207369  1601 log.cc:927] Time spent T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:12:40.105494   912 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.212136 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:40.192472  1386 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:40.231509  1603 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:12:40.231992 29007 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
W20260709 13:12:40.232234  1600 log.cc:927] Time spent T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:12:40.232746 29004 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:12:40.192857  1488 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:40.238101 29005 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:40.245662  1118 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:40.246446  1346 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:40.249447 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:40.252391  1518 raft_consensus.cc:1081] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Signalling peer befda4a6c8344c26b0f139e4e1686b3b to start an election
W20260709 13:12:40.253041 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 261850us
I20260709 13:12:40.289305 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:40.253685  1328 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:12:40.255142 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 237332us
I20260709 13:12:40.255302  1500 raft_consensus.cc:697] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:40.255635  1304 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:40.257524  1193 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:40.260933 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.278198 29004 pending_rounds.cc:85] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 8
I20260709 13:12:40.283814 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:40.313150 29004 pending_rounds.cc:107] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:12:40.313892 29007 raft_consensus.cc:3055] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:40.314230 29007 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:40.313199 29003 raft_consensus.cc:2376] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14. Re-sending same reply.
I20260709 13:12:40.314844 29159 raft_consensus.cc:2393] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:40.322855 29004 raft_consensus.cc:2889] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:40.319538 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:40.322594 29711 raft_consensus.cc:1217] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Deduplicated request from leader. Original: 9.10->[10.11-12.12]   Dedup: 10.11->[12.12-12.12]
I20260709 13:12:40.318142 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.343734 29711 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: [10.11-12.12]
I20260709 13:12:40.343883 29003 raft_consensus.cc:2410] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 13 because replica has last-logged OpId of term: 10 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 11.
I20260709 13:12:40.322230 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:40.344693 29159 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:40.319262 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:40.379164 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:40.379521 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:40.377516 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:40.273331  1214 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.318851 29007 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.347607 29085 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:12:40.388098 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44664.8us
I20260709 13:12:40.324080  1357 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:40.315670  1193 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:40.315500  1118 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:40.402280 29084 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:40.407016 29007 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:40.321040   912 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:40.325609  1510 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:12:40.326076  1501 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:40.332643  1518 consensus_queue.cc:1048] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:12:40.378037   909 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:12:40.317900  1386 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.399362  1193 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:40.435250 29004 raft_consensus.cc:1275] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:40.406101  1118 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:40.417583 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:40.419988  1214 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:40.420140 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:40.439808 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:40.420439 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:40.420944  1592 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.708s	user 0.002s	sys 0.006s
I20260709 13:12:40.421474 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:40.423650  1596 log.cc:927] Time spent T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.704s	user 0.000s	sys 0.009s
I20260709 13:12:40.424329  1499 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:40.401400  1357 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.319394  1488 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.449515  1500 consensus_queue.cc:237] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.462493 29713 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:40.441588  1118 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:40.467836 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:40.468468  1595 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.614s	user 0.009s	sys 0.009s
I20260709 13:12:40.468833 29083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:44428
I20260709 13:12:40.469566 29083 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:40.469907 29083 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:40.473011 29005 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:12:40.477351 29711 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:12:40.477430  1395 consensus_meta.cc:287] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.001s
I20260709 13:12:40.487419  1579 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:40.492589 29159 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:40.499667 29007 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:40.501751 29007 raft_consensus.cc:3055] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:40.502069 29007 raft_consensus.cc:740] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:40.502836 29007 consensus_queue.cc:260] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.503791 29007 raft_consensus.cc:3060] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:40.509838 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.445658  1376 consensus_queue.cc:1048] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:12:40.526540 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:40.453944  1499 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.536638 29007 raft_consensus.cc:1275] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:12:40.537209 29007 pending_rounds.cc:85] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 10
I20260709 13:12:40.551859 29007 pending_rounds.cc:107] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:12:40.552228 29007 raft_consensus.cc:2889] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:40.496224  1395 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.554929 29084 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:40.562721 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:40.564342 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:40.561417 29708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 703) took 1025 ms. Trace:
I20260709 13:12:40.524969  1579 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:12:40.573415 29158 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:40.564965 29083 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.575577 29080 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:40.567013 29708 rpcz_store.cc:276] 0709 13:12:39.529774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.536027 (+  6253us) service_pool.cc:224] Handling call
0709 13:12:39.536567 (+   540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:39.606871 (+ 70304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:39.606901 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:39.607419 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:12:39.709840 (+102421us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:39.709871 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.709919 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.423055 (+713136us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:40.549403 (+126348us) raft_consensus.cc:1685] finished
0709 13:12:40.549435 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.554993 (+  5558us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31958,"raft.queue_time_us":37144,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":21358,"spinlock_wait_cycles":47415680,"thread_start_us":96480,"threads_started":1,"wal-append.queue_time_us":111741}
I20260709 13:12:40.581845 29159 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:40.561631  1510 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.583077 29083 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:40.587430 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.572970   996 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:40.594472 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:40.595062 29150 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:40.595350 29150 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:40.600163   996 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:40.600479 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2545) took 1098 ms. Trace:
I20260709 13:12:40.601204   996 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.600900 29002 rpcz_store.cc:276] 0709 13:12:39.501721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.501885 (+   164us) service_pool.cc:224] Handling call
0709 13:12:39.502307 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:39.745991 (+243684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.600254 (+854263us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54003aaa54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:40.600420 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421217152}
I20260709 13:12:40.606824 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33750 (request call id 855) took 1002 ms. Trace:
I20260709 13:12:40.607319 29006 rpcz_store.cc:276] 0709 13:12:39.604593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.604731 (+   138us) service_pool.cc:224] Handling call
0709 13:12:40.606558 (+1001827us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54001d79d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:40.606693 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624554880}
I20260709 13:12:40.612823 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:40.616920 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:40.617502 32630 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:40.618040 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:40.619254 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.618151 32630 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:40.620115 32630 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:40.620200  1574 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:40.620846  1574 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:40.622134  1608 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:12:40.623548  1510 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:40.624584  1617 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:12:40.626820  1488 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:40.627390 29154 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:12:40.628896  1574 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:40.631071  1386 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:40.632165 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:40.633030  1591 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.000s
I20260709 13:12:40.633471  1591 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.633960 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2547) took 1126 ms. Trace:
I20260709 13:12:40.634341 29001 rpcz_store.cc:276] 0709 13:12:39.507782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.512179 (+  4397us) service_pool.cc:224] Handling call
0709 13:12:39.512643 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.151204 (+638561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.633747 (+482543us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54001d6fd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:40.633902 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1143835904}
I20260709 13:12:40.637039  1509 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.640875  1357 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:40.641556 29008 raft_consensus.cc:2368] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14.
W20260709 13:12:40.591427  1576 consensus_meta.cc:287] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.007s
I20260709 13:12:40.613965  1466 consensus_queue.cc:1048] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:12:40.594300  1618 log.cc:927] Time spent T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 13:12:40.649592 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.659464 29009 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 14: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:40.666585 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.642216  1509 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:40.672928 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:40.673023 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.670892 29150 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.683522 29150 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:40.684482 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:40.689505 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:40.690276 29159 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:40.679155  1488 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.695796  1509 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:40.696785  1193 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.698094 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:40.699141  1509 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:40.699249  1386 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.699856  1386 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.699759  1509 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:40.682914  1582 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.696830  1504 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:40.699008  1193 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:40.708647 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:40.711272 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 162543us
I20260709 13:12:40.707413  1582 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:40.714015  1627 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:12:40.714516  1582 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:40.715142  1509 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.718372 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:40.718735 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:40.703800  1386 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:40.721323 29159 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
W20260709 13:12:40.724020 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.724357 29083 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
W20260709 13:12:40.727494 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:40.732604  1626 log.cc:927] Time spent T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:12:40.732805 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:40.733420 29006 raft_consensus.cc:2376] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13. Re-sending same reply.
I20260709 13:12:40.734455 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.735507 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.736307 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.737032 29001 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 16: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:40.735320 29714 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:40.740484 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:40.741321 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:40.746940 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:40.747679 29010 raft_consensus.cc:2393] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:40.736320 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:40.748494 29007 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:12:40.751055 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:40.757021 29150 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:40.723944  1488 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.756141 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:40.764163 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:40.764798 29081 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:40.774384  1630 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:12:40.776947 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:40.797088 29710 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:40.794410  1500 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:40.800678  1500 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:40.808854  1500 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:40.808423 29151 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:40.798723  1591 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:40.836513 29157 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:40.834708 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:40.841288 29081 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
W20260709 13:12:40.849355 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:40.850878  1491 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:12:40.855932  1522 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:12:40.859280  1504 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:40.860961  1488 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:40.861474 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.843556  1511 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:40.868103  1118 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.872277  1244 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:40.875030 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:40.877524  1591 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:40.878823 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:40.899287 29156 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 14: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:40.914637  1591 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:40.915086  1591 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:40.911720 28975 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:40.917816 29152 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:40.907644  1491 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:40.925945  1118 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:40.919004  1509 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:40.915890  1591 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:40.933847  1491 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:40.948189 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2552) took 1096 ms. Trace:
I20260709 13:12:40.948931 29005 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:40.951958  1499 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:40.953456 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:40.948609 29009 rpcz_store.cc:276] 0709 13:12:39.851290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.968262 (+116972us) service_pool.cc:224] Handling call
0709 13:12:40.947947 (+979685us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54003ab454. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:40.948127 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561081600}
I20260709 13:12:41.021625  1492 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:41.022310  1492 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.027535  1193 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.028359  1193 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.042869  1492 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:41.043280  1193 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:41.091944 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.092674 29152 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:41.093873 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:41.094883  1492 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:41.095304  1492 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:41.095728  1492 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:41.130293   909 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.130982   909 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:40.921763  1304 consensus_queue.cc:1059] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.227s
I20260709 13:12:40.939210  1491 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.159346  1509 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:41.160883  1509 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.162976  1376 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.166226  1641 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 13:12:41.168159  1591 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:41.173401 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 216222us
W20260709 13:12:41.227752 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.282895 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.230280   909 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.278867  1304 consensus_queue.cc:1243] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:12:41.234683  1574 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:41.231492  1493 consensus_queue.cc:1059] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:12:41.236799  1567 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:41.309672 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.229933  1376 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.304536  1304 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.304348  1118 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:41.325275 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.325732 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.317889  1493 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:41.335009  1628 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.711s	user 0.013s	sys 0.004s
I20260709 13:12:41.335996  1296 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:41.336745 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.336526  1296 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.327452  1118 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.338653  1296 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:41.338780  1118 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:41.325834  1304 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.350706 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.351967  1304 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.354333  1376 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.355037  1493 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.358590 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.364137 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.364943 29152 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 10.
I20260709 13:12:41.317268  1574 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.366904 28975 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:41.364475 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.368063 29151 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:41.370031 28975 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:41.359637  1493 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.374116 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.374979 29150 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 14: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:41.366695  1500 consensus_meta.cc:287] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.556s	user 0.014s	sys 0.001s
I20260709 13:12:41.373731  1654 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:41.322474  1567 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.377133  1654 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.377942  1654 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:41.381309 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.381729 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.382018 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.387457 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:41.393414  1574 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:41.394264  1510 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:41.394452  1653 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.395010  1653 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.397229  1653 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.401305  1304 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.420660 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.444517 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.401922  1493 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.445981 29156 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:41.451555 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:41.454540 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2555) took 1494 ms. Trace:
I20260709 13:12:41.377257  1500 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.455001 29004 rpcz_store.cc:276] 0709 13:12:39.960309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.448767 (+488458us) service_pool.cc:224] Handling call
0709 13:12:40.463394 (+ 14627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.463567 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:41.396183 (+932616us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b18003fa5a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f41192393c7 00007f411584b9a7 00007f411585b0bf
0709 13:12:41.396357 (+   174us) log.cc:844] Serialized 14 byte log entry
0709 13:12:41.454163 (+ 57806us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:41.454225 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:41.454280 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.454384 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:41.454496 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6930,"prepare.queue_time_us":50794,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":5300,"spinlock_wait_cycles":8731008,"thread_start_us":86103,"threads_started":2,"wal-append.queue_time_us":46688}
I20260709 13:12:41.426570  1304 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.477453  1395 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:41.452401  1493 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.485687 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1977) took 1090 ms. Trace:
I20260709 13:12:41.487188 29158 rpcz_store.cc:276] 0709 13:12:40.394908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.395055 (+   147us) service_pool.cc:224] Handling call
0709 13:12:40.395333 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.578572 (+183239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.485350 (+906778us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000951d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:41.485579 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406876928}
I20260709 13:12:41.510896  1304 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.514102 29078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2273) took 1097 ms. Trace:
I20260709 13:12:41.514613 29078 rpcz_store.cc:276] 0709 13:12:40.416492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.416612 (+   120us) service_pool.cc:224] Handling call
0709 13:12:40.435764 (+ 19152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.453633 (+ 17869us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:40.474649 (+ 21016us) log.cc:844] Serialized 14 byte log entry
0709 13:12:40.531849 (+ 57200us) log.cc:844] Serialized 14 byte log entry
0709 13:12:40.560583 (+ 28734us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:40.560642 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:40.560680 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.513834 (+953154us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b54000249d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:41.513889 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:41.514026 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55283,"prepare.queue_time_us":26943,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":146036,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":16245,"spinlock_wait_cycles":151117184,"thread_start_us":43698,"threads_started":1,"wal-append.queue_time_us":112252}
I20260709 13:12:41.516386  1661 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:41.517953  1493 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:41.523042 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.523730 29712 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:41.535343  1581 raft_consensus.cc:3055] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:41.535750  1581 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:41.536717  1653 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:41.541674  1661 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.543566  1492 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:41.556779  1629 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.957s	user 0.005s	sys 0.011s
I20260709 13:12:41.554602  1654 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.563529 29005 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:41.571503  1500 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:41.572216  1567 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.574882  1386 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:41.575963  1500 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.576541  1500 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.578542  1500 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:41.579406  1123 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.579978  1123 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.582067  1123 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:41.582247  1654 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.582978 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.582494  1123 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.584218  1123 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.585045 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.585407  1492 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:41.585654 29158 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:41.586109  1123 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:41.586601 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.587726 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:41.588305 29151 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.588375  1500 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:41.589272  1395 consensus_queue.cc:1059] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:41.590453  1510 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:41.590811  1510 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:41.591741  1510 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.593187  1510 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:41.593103  1659 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:41.593472  1504 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.594092  1504 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.597363  1576 raft_consensus.cc:3055] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:41.600502 28975 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:41.597836  1576 raft_consensus.cc:740] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:41.609946 28975 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:41.611088  1502 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.611656  1502 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.612077  1573 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:41.612479  1573 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:41.612805  1573 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:41.613796  1502 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.614297  1502 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.614703  1502 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.616204  1654 consensus_queue.cc:1059] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:41.616326  1504 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.619374 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:41.640632 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.644054 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.645424 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.647199 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.647904 29078 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:41.651754 29151 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:41.654764 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.658504 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.659205 29084 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:12:41.665411 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.666836  1666 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.667420  1666 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.671169 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.671792 29082 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.672629 29081 raft_consensus.cc:1240] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:41.673347 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.666831  1615 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.677477 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.677982 29084 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:41.682188 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.678630  1615 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.686892  1573 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.693435 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.694093 29081 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.696254  1668 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.696782  1668 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.697741 29077 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:12:41.700312  1624 log.cc:927] Time spent T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.126s	user 0.005s	sys 0.012s
I20260709 13:12:41.701417 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 1972) took 1454 ms. Trace:
I20260709 13:12:41.701771 29153 rpcz_store.cc:276] 0709 13:12:40.246957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.324202 (+ 77245us) service_pool.cc:224] Handling call
0709 13:12:40.324499 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:40.324762 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:40.324800 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:40.325323 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:12:40.405496 (+ 80173us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:40.405537 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.405597 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:41.699739 (+1294142us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:41.701125 (+  1386us) raft_consensus.cc:1685] finished
0709 13:12:41.701148 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:41.701359 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20702080,"thread_start_us":79797,"threads_started":1,"wal-append.queue_time_us":183016}
I20260709 13:12:41.704509 29714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 592) took 1326 ms. Trace:
I20260709 13:12:41.704854 29714 rpcz_store.cc:276] 0709 13:12:40.377675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.382743 (+  5068us) service_pool.cc:224] Handling call
0709 13:12:40.735114 (+352371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.747631 (+ 12517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.704314 (+956683us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b540021f550. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:41.704437 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768500096}
W20260709 13:12:41.706806 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 988033us
I20260709 13:12:41.708757 29712 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:41.709055  1214 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.711277  1214 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.712117  1666 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.713128  1668 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.713300  1214 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.717797 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.718437 29716 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:41.679396  1502 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.721138 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.724937 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.726454 29717 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:41.726999 29076 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:12:41.727941 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.728591 29084 raft_consensus.cc:1275] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:12:41.732522 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.735380 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:41.737840 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:41.738013 29082 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:41.744032 29077 raft_consensus.cc:1240] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 10. Current term is 11. Ops: [8.6-10.8]
I20260709 13:12:41.744354 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.744987 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.745425 29083 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:41.746203 29710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47384 (request call id 919) took 1041 ms. Trace:
I20260709 13:12:41.746572 29710 rpcz_store.cc:276] 0709 13:12:40.704835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.705001 (+   166us) service_pool.cc:224] Handling call
0709 13:12:41.746040 (+1041039us) spinlock_profiling.cc:243] Waited 78.1 ms on lock 0x7b5400650254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:41.746125 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179616000}
I20260709 13:12:41.747028 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.747605 29078 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:41.738986 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.749606 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.750447 29078 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
W20260709 13:12:41.752076 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.754566 29076 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: [10.8-11.9]
I20260709 13:12:41.755836 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.756577 29084 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:41.738175 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:41.765758 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.719287  1386 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:41.770872 29081 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:41.773926 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.747874  1123 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:41.779968 29076 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:41.780448 29082 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:41.789915 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.772688  1552 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:41.791206 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.791316 29082 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
W20260709 13:12:41.792213 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55195.4us
W20260709 13:12:41.793017  1670 log.cc:927] Time spent T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:12:41.795226 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:41.796620 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.798808 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2276) took 1283 ms. Trace:
I20260709 13:12:41.800957 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.801530 29076 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:41.802050  1653 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.802506  1653 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:41.780691  1503 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:41.782130  1123 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:41.799096 29080 rpcz_store.cc:276] 0709 13:12:40.515877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.519624 (+  3747us) service_pool.cc:224] Handling call
0709 13:12:40.532328 (+ 12704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.582859 (+ 50531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.798575 (+1215716us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b54001211d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:41.798738 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97684736}
W20260709 13:12:41.790329  1580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.188s	user 0.019s	sys 0.024s
I20260709 13:12:41.772228  1622 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:12:41.796309  1573 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:41.807845 29157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 1969) took 1704 ms (client timeout 2162 ms). Trace:
W20260709 13:12:41.810518 29157 rpcz_store.cc:269] 0709 13:12:40.094036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.129588 (+ 35552us) service_pool.cc:224] Handling call
0709 13:12:41.797969 (+1668381us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b540013e554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:41.798088 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1031956352}
I20260709 13:12:41.812229 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.813547 29083 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:41.814769 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.818418 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.818696  1362 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:41.818981 29077 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 8.
I20260709 13:12:41.799687  1615 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.824747 29076 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:41.808329  1503 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:41.809682  1123 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:41.829160 29084 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
W20260709 13:12:41.810408  1580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.019s	sys 0.025s
I20260709 13:12:41.815887  1460 raft_consensus.cc:2764] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:12:41.831761 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:41.832659 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.833215 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.834730 29080 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:41.833674 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.834949 29083 raft_consensus.cc:1217] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Deduplicated request from leader. Original: 10.10->[12.11-12.11]   Dedup: 12.11->[]
I20260709 13:12:41.835505 29083 raft_consensus.cc:1240] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:12:41.849663 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:41.852759 29081 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:12:41.831498  1503 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:41.850450 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.854478 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:41.863121 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.863135 29083 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:41.862236  1622 raft_consensus.cc:2764] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:12:41.849615  1362 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:41.869139  1362 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.871452  1362 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:41.879274 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.876727  1502 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:41.861330 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:41.881592 29081 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 14: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:41.887120 29083 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:41.852823  1615 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:41.891032 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:41.879171  1669 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:41.896070 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46467.8us
I20260709 13:12:41.897217 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:41.921159 29080 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:41.922605 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:41.925163 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 360487us
I20260709 13:12:41.925872 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:41.927157 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:41.928229  1567 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:12:41.928580 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.928750  1567 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:41.929879  1503 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.928270  1346 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:41.931844 29084 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:41.929765  1493 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:41.932343  1460 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:41.932391  1503 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:41.932906  1460 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:41.933359  1460 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:41.936800 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 326879us
I20260709 13:12:41.936924  1509 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:41.937530  1509 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:41.937666 28975 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:41.938002  1509 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:41.938777 28975 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:41.940237  1501 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:41.940619  1573 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:41.940727  1501 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:41.941252  1573 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:41.941833  1501 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.942178  1573 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.943773  1493 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:41.943861  1655 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.944171  1491 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.944449  1493 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.944514  1655 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.946393  1513 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.924131  1118 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.930727 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:41.927934 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:41.939977 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.967002 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.967509 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.970142 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52310.6us
I20260709 13:12:41.971582 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:41.974323 29152 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:41.974627  1581 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.975708  1581 raft_consensus.cc:3060] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.959378  1118 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.976538 29151 raft_consensus.cc:1240] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:41.947139  1491 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.978027  1118 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.978384  1653 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.979164  1491 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.946861  1513 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.952132  1591 raft_consensus.cc:3055] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:12:41.980626  1673 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:12:41.976564 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.967288  1552 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:41.981545 29156 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:41.981716  1552 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:41.980268  1591 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:41.981114  1513 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.983788  1552 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:41.957815  1576 consensus_queue.cc:260] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:41.984429 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:42.016341 29156 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:41.992828 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.000576  1654 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:42.002841 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.003365 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.971026 29050 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:42.018281 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 47320.4us
I20260709 13:12:41.954743  1655 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.022881 28975 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:42.024485 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:41.971714  1502 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:41.981135  1659 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:41.953869  1509 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:42.034755 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 104103us
I20260709 13:12:42.025529 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:42.045547 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:42.046873  1491 consensus_queue.cc:1175] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "243d4ae4a4ae48b2979e32874472d256" responder_term: 12 status { last_received { term: 12 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:42.024837  1668 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:42.054375 29151 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:42.032424  1655 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:42.028613  1622 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:12:42.046875  1576 raft_consensus.cc:3060] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:42.022533  1654 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.056783  1214 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:42.062395 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.055365 29125 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.058595 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.063491 29008 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:42.059362 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:41.929729  1567 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.064424 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.064716 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.060801 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.065361 29150 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:12:42.065469 29007 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:42.067605 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.064999 29158 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:42.069942 29084 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:42.028954  1513 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:12:42.072773 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:42.075446 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.076550 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.052814  1653 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:42.073426 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.078904 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:42.079397 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.080462 29009 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:42.081138 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:42.058128  1655 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.082405 29084 raft_consensus.cc:2410] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:12:42.080922 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.083964 29154 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:42.060708  1493 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.080830   912 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:42.089540   912 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:42.089947   912 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:42.092913  1591 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.093956  1591 raft_consensus.cc:3060] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.060352  1668 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:42.098634  1668 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.100589  1669 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:42.078593  1666 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:12:42.078859  1214 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:42.103654  1214 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.069641  1615 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 13:12:42.105574 29683 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.66:35939 timed out after 2.221s (SENT) [suppressed 22 similar messages]
I20260709 13:12:42.108506 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.086196  1395 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:42.125102 29050 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:42.127516 29156 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:42.119238 29085 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:42.129400 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:42.134783 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 73704.4us
I20260709 13:12:42.127204  1395 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:42.142519 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.121223  1655 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:42.132638  1328 raft_consensus.cc:2804] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:42.148800 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.134671  1622 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:42.159085   988 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:42.160704 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.161484 29001 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 14: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:42.162187  1460 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.160828  1661 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:42.135277  1615 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:42.158375 29009 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:42.163937 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:42.168967 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63425us
I20260709 13:12:42.145001  1291 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:42.173892 29005 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:42.146679  1304 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:42.159641   988 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.181766 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:42.180447  1678 log.cc:927] Time spent T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:12:42.187585 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:42.187915 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.178248  1510 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:42.174587  1291 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:42.190361 29078 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:42.147279  1123 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.193214  1151 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:42.171862  1507 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:12:42.183768  1304 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.174642  1666 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:12:42.212797 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.226902 29159 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:12:42.227133 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.227566 29157 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 14: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:42.184242  1362 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:42.179095  1504 consensus_queue.cc:1048] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:12:42.204711 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.204308 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.215562  1667 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:42.222090 29048 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:42.195438  1510 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.241861 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:42.242035 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.202957 29125 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.228641  1492 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:42.233669 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.234247 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.194957  1622 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:42.238013 29006 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:42.234061  1291 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.240582 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.244671 29002 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:42.245913 29125 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.204478  1509 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.214345  1615 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:42.227878  1328 raft_consensus.cc:697] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:42.257840 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:42.270539 29004 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:42.275614 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:42.275964 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.212333  1579 consensus_queue.cc:1048] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:12:42.247498  1510 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.306308 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:42.313961 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56206.1us
I20260709 13:12:42.314651 29006 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:12:42.316386 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 94371.8us
I20260709 13:12:42.252904   909 raft_consensus.cc:2804] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:42.319808 29004 raft_consensus.cc:1240] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:42.333503 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.337901 29155 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:42.277235  1123 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:42.244293  1362 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:42.343983  1683 log.cc:927] Time spent T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:12:42.360493 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.317191  1214 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:42.244635  1667 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.350062 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:42.353224 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.374600 29004 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 14: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:42.353495 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:42.249781   912 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.375136 29153 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.361441 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:42.378153 29081 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:42.273669  1492 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:42.371330 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.356062 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.364604 29078 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 9.
I20260709 13:12:42.342689  1291 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:42.315697  1690 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:42.316998  1151 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.052s
W20260709 13:12:42.395841 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 78357.8us
I20260709 13:12:42.396426 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.354975  1552 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.397727 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:42.391839 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.408020 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.398348 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.409133 29079 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:42.270098  1622 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.401737 29159 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:42.345718  1693 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:42.404601 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:42.355888  1460 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:42.429692  1692 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:12:42.407056 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.399976  1690 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.430902 29084 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:42.431145  1690 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:42.432232  1214 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:42.418051  1622 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.420429  1492 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.420495   988 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:42.435772  1510 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.422055   912 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:42.422945 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.424891  1328 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.428289 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.439718 29077 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:42.414113  1693 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.428979   996 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:12:42.440663  1685 log.cc:927] Time spent T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:12:42.440873   996 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:42.441305   996 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:42.441917  1697 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:42.420717  1304 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.442348  1697 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.442739  1697 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:42.444156  1123 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:42.444514  1362 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:42.444711  1690 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.447801  1690 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:42.450542   996 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.453612  1510 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:42.454455   996 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:42.456990 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.414422  1666 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:12:42.457966 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.458289 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:42.458655  1291 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.413805  1552 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.459219  1291 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.460831  1552 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:42.461583 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.461974  1291 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.462639 29009 raft_consensus.cc:1275] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:42.454543  1667 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.472728 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.465994 29007 raft_consensus.cc:2468] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:42.467413 29153 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:42.465863 29008 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:42.472781 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:42.468318 29003 raft_consensus.cc:2468] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:42.474486 29048 leader_election.cc:400] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14.
I20260709 13:12:42.474845 29048 leader_election.cc:403] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:12:42.473841  1579 raft_consensus.cc:2764] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:12:42.477236 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.486126 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:42.486091 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:42.492261 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.487782  1460 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14.
I20260709 13:12:42.440681  1693 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:42.499678 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:42.500398 29713 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:42.498935 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 83450.7us
I20260709 13:12:42.501715 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:42.513993  1616 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:42.514475  1616 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:42.514396  1655 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.494745  1552 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.516597  1552 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.518268 29001 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 14: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:42.518644  1552 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:42.519044 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.486946  1151 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:42.520583  1616 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.521886  1616 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.526765  1693 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.530409 29713 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:42.530999 28975 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:42.534245  1304 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:42.535465  1362 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:42.537079  1655 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.537573  1155 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:42.539492  1466 raft_consensus.cc:3055] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:42.539849  1466 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:42.540661  1466 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.541796  1466 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:42.547950 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:42.552961 28975 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:42.555742 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:42.566237 29152 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:42.567417  1710 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:42.567505  1655 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:42.568528  1655 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:42.574950 29156 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:42.578281 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.572225  1710 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:42.581080 29152 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:42.581643  1710 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.582412  1693 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:42.587971  1666 consensus_queue.cc:1059] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:42.588970  1702 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:42.589395  1702 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:42.590106  1702 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.591233  1702 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:42.598644 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:42.599277 29713 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:42.570092  1576 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:42.622332 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 134037us
W20260709 13:12:42.623546 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:42.607761 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 257795us
I20260709 13:12:42.624631 29713 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:42.569378  1655 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.565716  1460 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:12:42.637302 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:42.638082 29085 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:42.640717 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.641561 29006 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:42.642354 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:42.643728 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:42.661446 29155 raft_consensus.cc:1240] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
W20260709 13:12:42.663213 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 128080us
I20260709 13:12:42.669812 29713 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:42.669324 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:42.674677 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.679041 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:42.606834   909 raft_consensus.cc:697] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:42.682299 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:42.686115 29715 raft_consensus.cc:2393] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 14: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:42.633888  1576 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:42.698040 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:42.698668 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:42.701220 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:42.701786 29082 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:42.702570  1328 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:42.707026 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:42.707621 29080 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.708707 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:42.712702  1690 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:42.713073  1690 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:42.713435  1690 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:42.716223   988 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:42.716665   988 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:42.716990   988 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:42.719964  1693 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:42.721874  1693 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.722340  1693 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.724306 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.724895 29084 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:42.720112  1123 consensus_queue.cc:1059] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:12:42.720427 29080 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:42.719452 29082 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:42.636734  1510 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:42.730180 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.736670 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:42.737736 29715 raft_consensus.cc:2410] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:12:42.738485 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.726814  1693 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:42.742065 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:42.686839  1706 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:42.756983 29085 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:42.743566  1123 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:42.770709 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.760607   996 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:12:42.774097 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.774606  1576 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.798960 29085 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:42.760712  1654 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:42.796309  1123 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:42.755048  1706 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.836532 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.837275 29081 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 14: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:42.838083  1706 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:42.838208 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.849782 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.855651 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:42.872319 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 130439us
I20260709 13:12:42.828603   909 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.842706  1697 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.887046 29151 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 14: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:12:42.896855 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 99062.5us
I20260709 13:12:42.890059  1518 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.898146  1518 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.897205  1669 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:42.900405  1518 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:42.890302  1328 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:42.881385  1123 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.914088 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:42.901790  1669 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.923616  1328 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.924652  1720 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:42.925197  1720 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.925609  1720 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.926111 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:42.929172  1690 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:42.930730 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:42.930948  1718 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:12:42.931165 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:42.931504 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.934219 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:42.936208  1500 consensus_queue.cc:1048] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:42.937937  1723 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:42.938498  1723 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:42.940426  1717 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:12:42.940330 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:42.941250 29125 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.942029 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:42.942729  1721 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:42.942960  1719 log.cc:927] Time spent T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:12:42.943255  1721 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:42.943499 29083 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:42.944002  1706 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:42.944659 29083 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:42.946395 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:42.939379  1723 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.909283   988 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:42.949889 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.953701 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.954245 29008 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:42.964073 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:42.955098 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.965126 29006 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:42.954190 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.966382 29004 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.955327 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.967402 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.964169 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.955358 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.969719 29001 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:42.970741 29713 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:42.973479 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:42.944190  1721 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:42.990257 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:42.990972 29003 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:42.953728  1707 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:42.994733 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:12:42.995370 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.964102 29714 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:42.998764 29122 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:43.002631 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:42.960580  1579 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:12:42.961220  1151 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:12:43.037958 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:43.040304 29009 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:43.044811 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:42.995047  1707 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.997208  1706 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:43.054095 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.049211 29004 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:42.958947  1395 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.096109 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:43.107555 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.108016 29004 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:43.042194  1697 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:43.130630  1386 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:43.131094  1386 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:43.132674  1697 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:43.133421  1697 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:43.133791  1697 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:43.134112  1697 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:43.132987  1386 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.134790  1386 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:43.147431 29051 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:43.053960  1707 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:43.148943 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:43.156555 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:43.159612 29009 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:43.161975 29004 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
W20260709 13:12:43.188056 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.193706 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:43.149516  1702 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:43.196260 29004 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:43.194548 29009 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 14: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:43.183314  1690 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:43.202431 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 282862us
I20260709 13:12:43.184417  1616 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:12:43.205123 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.191709  1395 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:43.210144 29683 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:43.213083 29683 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:43.214190 29683 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:43.219959 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2568) took 1132 ms. Trace:
I20260709 13:12:43.220387 29007 rpcz_store.cc:276] 0709 13:12:42.084183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.108254 (+ 24071us) service_pool.cc:224] Handling call
0709 13:12:43.214960 (+1106706us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54001856d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:43.215089 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632729344}
W20260709 13:12:43.225204 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 222667us
I20260709 13:12:43.219650  1715 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:43.236277 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:43.226433  1362 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:43.203708  1706 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.263823 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:43.240675  1728 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:43.265149 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:43.246977  1500 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:12:43.290512 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 73482.8us
I20260709 13:12:43.300896  1616 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:43.301455  1616 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.303442  1616 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:43.272912  1734 consensus_queue.cc:1059] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:12:43.314931  1502 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:43.316488   996 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:43.316952   996 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:43.317551  1579 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.318051  1579 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:43.281668  1731 log.cc:927] Time spent T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.175s	user 0.001s	sys 0.001s
I20260709 13:12:43.282146  1728 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:43.328648 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.331997 29006 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:43.315475  1720 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:43.283511  1723 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:43.341138 29001 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:43.293643  1690 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:12:43.324026  1732 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:12:43.347157 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:43.343531 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:43.326972  1500 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:43.320948   909 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.320004  1579 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:43.348598 29006 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:43.322038  1503 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:43.374569 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.378835 29715 raft_consensus.cc:2393] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 13: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:43.334542  1616 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:43.326892  1728 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:43.344950  1734 consensus_queue.cc:1243] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer 06ae4197fd784525b695a9b16c8094ea log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:12:43.315477  1502 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.379469  1500 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:43.379237  1723 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:43.390056  1503 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.410287  1503 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:43.389211  1616 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.411759  1502 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:43.416318  1707 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.393076   909 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:43.421466 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 71385us
W20260709 13:12:43.565466 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 188102us
I20260709 13:12:43.566741 29122 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:43.562986 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 219500us
I20260709 13:12:43.575124 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:43.580497 29154 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:12:43.500862  1716 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:43.608152  1750 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:43.610962  1745 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:43.616680  1752 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:12:43.625865  1720 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:43.627058  1740 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:43.627614  1740 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:43.629667  1740 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:43.611516  1745 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.633798  1616 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:43.592669  1743 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
W20260709 13:12:43.593571  1737 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
W20260709 13:12:43.579571  1722 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.590s	user 0.000s	sys 0.003s
I20260709 13:12:43.631512   996 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.597577  1706 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:43.576978  1503 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.619773  1697 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:43.784564 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 209518us
W20260709 13:12:43.875645  1714 log.cc:927] Time spent T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.994s	user 0.001s	sys 0.001s
I20260709 13:12:43.875932  1576 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:43.876526  1576 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:43.877085 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2305) took 1377 ms. Trace:
I20260709 13:12:43.878513  1576 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:43.877573 29079 rpcz_store.cc:276] 0709 13:12:42.499257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.504633 (+  5376us) service_pool.cc:224] Handling call
0709 13:12:42.504950 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:42.512575 (+  7625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.512609 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:42.513084 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:12:42.745891 (+232807us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:42.745940 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.746015 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:43.875032 (+1129017us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:43.876775 (+  1743us) raft_consensus.cc:1685] finished
0709 13:12:43.876799 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.877018 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7159,"raft.queue_time_us":8288,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"spinlock_wait_cycles":7851392,"thread_start_us":232437,"threads_started":1,"wal-append.queue_time_us":336349}
I20260709 13:12:43.879081 29709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 604) took 1234 ms. Trace:
W20260709 13:12:43.800829  1722 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:12:43.780432  1690 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:43.863754  1386 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:43.871132  1503 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.648736  1723 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:43.583022  1502 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:43.868321  1395 consensus_meta.cc:287] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.006s
I20260709 13:12:43.886504 29082 raft_consensus.cc:1240] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:43.906564 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:43.888023 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:43.886780 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:43.882583  1492 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:43.895241  1707 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:43.894959  1386 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:43.896055 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:43.891888  1740 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.879631  1244 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:43.920883  1740 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:43.921348  1244 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:43.898217 29079 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:43.879494 29709 rpcz_store.cc:276] 0709 13:12:42.645053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.645220 (+   167us) service_pool.cc:224] Handling call
0709 13:12:42.673598 (+ 28378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:42.865391 (+191793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.865416 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:42.877478 (+ 12062us) log.cc:844] Serialized 26 byte log entry
0709 13:12:42.994593 (+117115us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:43.251705 (+257112us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.877774 (+626069us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b180020f320. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157bc733 00007f4115841dbf 00007f411584083e
0709 13:12:43.878687 (+   913us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b54005e2ca0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157bc733 00007f4115841dbf 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c
0709 13:12:43.878754 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.878847 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:43.878889 (+    42us) raft_consensus.cc:1685] finished
0709 13:12:43.878901 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.879025 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44803,"raft.queue_time_us":142835,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":4352,"spinlock_wait_cycles":97939968,"thread_start_us":521346,"threads_started":3,"wal-append.queue_time_us":337575}
I20260709 13:12:43.908531 29156 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
W20260709 13:12:43.910985 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 341802us
I20260709 13:12:43.911065 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:44.050172 29082 raft_consensus.cc:2410] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:12:44.050948 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:43.873026  1616 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:43.890159  1669 consensus_meta.cc:287] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.939s	user 0.006s	sys 0.025s
I20260709 13:12:43.880272  1706 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:43.916874 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:44.062458 29077 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:43.909336  1492 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:44.041314 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:44.044431 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2595) took 1082 ms. Trace:
I20260709 13:12:44.064471 29085 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:44.071026  1745 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:44.071388  1492 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:43.909821  1728 consensus_meta.cc:287] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.018s
I20260709 13:12:43.910514  1502 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.072355   909 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:44.074270  1502 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.074383 29085 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:44.071784  1740 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.042055  1716 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:44.075053  1706 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.056965  1760 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.057297 29082 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:44.072585  1728 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.078694  1728 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:44.044210 28976 connection.cc:256] RPC call timeout handler was delayed by -1.8092086980000204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
I20260709 13:12:44.063990  1616 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.043247  1707 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:44.079958  1707 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.080761  1503 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.080972  1616 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.081979  1707 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:44.041383  1500 consensus_meta.cc:287] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.635s	user 0.008s	sys 0.009s
W20260709 13:12:44.082486 28976 connection.cc:256] RPC call timeout handler was delayed by -1.72010869799999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
I20260709 13:12:44.082590  1500 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.012367  1690 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.082230 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:44.088146 29082 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:44.068559 29008 rpcz_store.cc:276] 0709 13:12:42.962444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.968819 (+  6375us) service_pool.cc:224] Handling call
0709 13:12:42.969118 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:42.976166 (+  7048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.986941 (+ 10775us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:42.992468 (+  5527us) log.cc:844] Serialized 26 byte log entry
0709 13:12:43.258764 (+266296us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:43.258818 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.258905 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:44.044225 (+785320us) raft_consensus.cc:1685] finished
0709 13:12:44.044244 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:44.044367 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":12868,"thread_start_us":214303,"threads_started":1,"wal-append.queue_time_us":538045}
I20260709 13:12:44.072562  1492 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:44.088814 29077 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:44.091868 29079 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:12:44.046543  1395 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:44.082343  1386 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.092775 29152 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:44.092828  1395 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.093488  1765 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:44.093758  1500 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:44.094030  1765 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.090576  1760 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.095554  1395 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.058473  1623 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.064329  1669 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:44.100802 29714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47388 (request call id 603) took 1456 ms (client timeout 1541 ms). Trace:
W20260709 13:12:44.110574 29714 rpcz_store.cc:269] 0709 13:12:42.644755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.644940 (+   185us) service_pool.cc:224] Handling call
0709 13:12:44.100633 (+1455693us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54005b8454. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:44.100737 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625952128}
W20260709 13:12:44.101876 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.101661 29083 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:44.103341 29710 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:44.114638 29085 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:12:44.214885 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.102496  1669 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.102767   909 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:44.102895  1740 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:44.096024  1765 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:44.296392 29082 raft_consensus.cc:2468] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:44.100090  1623 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.117242   996 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:44.322551 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:44.327512 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:44.303894  1697 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:44.343709 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:44.344645 29708 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:44.355504 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:44.358641 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:44.337930  1669 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.304613   909 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.311614  1740 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:44.376533 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 49094.9us
I20260709 13:12:44.387280 29155 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:44.365792  1669 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.380878   996 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:44.400660 29157 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:12:44.402145 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:44.393930   988 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:44.434671  1386 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:44.505479 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 422209us
W20260709 13:12:44.511448 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6985086980000688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 13:12:44.512143 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6895086980000542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 13:12:44.516346 28976 connection.cc:256] RPC call timeout handler was delayed by -1.663408698000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 13:12:44.516850 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6325608509999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:12:44.517238 28976 connection.cc:256] RPC call timeout handler was delayed by -1.6139086980000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
W20260709 13:12:44.517726 28976 connection.cc:256] RPC call timeout handler was delayed by -1.2224086979999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2681s for a response
W20260709 13:12:44.518102 28976 connection.cc:256] RPC call timeout handler was delayed by -1.1584228620000658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:12:44.518682  1623 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.506304  1386 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:44.524236  1772 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:12:44.528625   909 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.564857  1669 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.571357 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2313) took 2024 ms. Trace:
W20260709 13:12:44.572368 29051 leader_election.cc:336] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Timed out: RequestConsensusVote RPC to 127.27.63.68:44167 timed out after 1.694s (SENT)
I20260709 13:12:44.571764 29080 rpcz_store.cc:276] 0709 13:12:42.546780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.734671 (+187891us) service_pool.cc:224] Handling call
0709 13:12:42.735097 (+   426us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:42.753261 (+ 18164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.753290 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:12:42.760023 (+  6733us) log.cc:844] Serialized 78 byte log entry
0709 13:12:42.923684 (+163661us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b480015c258. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411a76aaa1 00007f411a761b37 00007f411a7619ca 00007f411a85926f 00007f411a858f37 00007f411a86d55c 00007f411a86d508 00007f411a86d488 00007f411a86d443 00007f411a86c6ec 00007f4119796a96
0709 13:12:44.043756 (+1120072us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b18001c1a40. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:12:44.311433 (+267677us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:44.424494 (+113061us) log.cc:844] Serialized 14 byte log entry
0709 13:12:44.433246 (+  8752us) log.cc:844] Serialized 14 byte log entry
0709 13:12:44.433768 (+   522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.433843 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:44.571108 (+137265us) raft_consensus.cc:1685] finished
0709 13:12:44.571132 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:44.571298 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18077,"prepare.queue_time_us":263818,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":32149,"raft.run_cpu_time_us":1572,"raft.run_wall_time_us":217777,"spinlock_wait_cycles":72702208,"thread_start_us":351454,"threads_started":2,"wal-append.queue_time_us":1611011}
I20260709 13:12:44.622253 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.623095 29157 raft_consensus.cc:1240] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:44.623826 29153 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 13:12:44.626998  1746 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 1.234s	user 0.013s	sys 0.017s
W20260709 13:12:44.628374 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.636088 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 110346us
I20260709 13:12:44.637640 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:44.640470 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.643591  1785 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.647248  1782 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:12:44.653439  1386 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.654544 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2675) took 1321 ms. Trace:
I20260709 13:12:44.654963  1782 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:44.654911 29154 rpcz_store.cc:276] 0709 13:12:43.333110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.333217 (+   107us) service_pool.cc:224] Handling call
0709 13:12:43.354789 (+ 21572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:43.583425 (+228636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.654313 (+1070888us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b540013d154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:44.654490 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518441728}
I20260709 13:12:44.655658  1782 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:44.658685  1760 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:44.656296  1782 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:44.664584  1736 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.328s	user 0.002s	sys 0.000s
I20260709 13:12:44.667752  1782 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:44.668275 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.668814 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.675956 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.676805 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.677086 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.677625 29157 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:44.677891 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.678368 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.678650 29154 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:12:44.679720 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.680258 29151 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:44.681049 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.682102 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.682679 29158 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:44.683514 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.685501  1362 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:44.685990  1362 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.688023  1362 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.690428  1733 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:12:44.686007 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.690920  1733 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:44.692368 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:44.693292 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:44.693687 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:44.693882 29717 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:44.644196  1785 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.694856 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:44.695981 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:44.679417 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:44.696215  1785 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.689273 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2673) took 1356 ms. Trace:
I20260709 13:12:44.698040 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.697834 29155 rpcz_store.cc:276] 0709 13:12:43.332548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.332648 (+   100us) service_pool.cc:224] Handling call
0709 13:12:43.348946 (+ 16298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.389194 (+1040248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.689066 (+299872us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400054e54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:44.689223 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2387322240}
I20260709 13:12:44.691713  1733 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.703509 29008 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:44.712028 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:44.679404 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:44.716189 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.717012  1518 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.718189 29051 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:44.722194  1518 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.723033 29002 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:12:44.723906 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.724392 29683 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:44.724818  1616 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.725294  1616 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.726361  1727 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:44.727255  1616 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.727666 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:44.728816 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.729357 29006 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:44.724478 29005 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:44.722210  1669 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:44.734036  1669 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.734392  1669 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:44.736436 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.737627 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.738368  1623 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.692461  1784 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:44.744333 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.745039 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.745692 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.746124 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.754004 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 116441us
I20260709 13:12:44.755663 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:44.763031 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.763136 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.766225 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:44.738816  1623 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.768147 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.768350 29004 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:44.768309 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:44.770726 29006 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:44.739125  1386 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:44.742295  1727 consensus_queue.cc:1059] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:12:44.772253 29004 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:44.772387 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:44.774995 29002 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:12:44.743083  1615 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:44.779147 29152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2674) took 1446 ms. Trace:
I20260709 13:12:44.775368 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.779649 29152 rpcz_store.cc:276] 0709 13:12:43.332817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.332940 (+   123us) service_pool.cc:224] Handling call
0709 13:12:43.333174 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.093306 (+760132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.778873 (+685567us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b54003106d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:44.779071 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1746459904}
I20260709 13:12:44.799729  1787 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:44.800518  1787 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.803027  1787 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.803547  1787 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.805794  1791 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.806291  1791 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.807668 29002 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
W20260709 13:12:44.808688 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 47476.2us
I20260709 13:12:44.809505 29683 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:44.821557 29006 raft_consensus.cc:2468] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:44.746512  1759 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:12:44.828003 29005 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:44.837280 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.816771  1761 raft_consensus.cc:2804] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:44.841490  1785 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:44.761240  1784 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.776230  1386 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.854666 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.855544 29005 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:44.830449  1749 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.803983  1787 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.829082  1786 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:44.828905  1492 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:12:44.813578  1615 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.858028  1761 raft_consensus.cc:697] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:12:44.895042 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 62417.8us
I20260709 13:12:44.867058  1790 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:44.868834  1791 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.870923  1518 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:44.903498 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 60270.4us
I20260709 13:12:44.903855 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:44.905370 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.902944 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:44.915251 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 159649us
I20260709 13:12:44.886286  1616 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:44.917332 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:44.872316  1782 consensus_queue.cc:1059] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:12:44.864107  1749 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.899528  1738 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:12:44.915540  1792 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:12:44.919538  1791 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.960943  1795 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:44.966221  1795 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.961277  1759 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:44.968590  1759 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.969002  1759 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:44.961481  1518 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:44.971909  1518 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.972358  1518 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:44.915020  1770 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:12:44.977734  1386 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.978394  1386 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.978912  1386 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.980072  1616 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:44.982522  1616 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.908063  1790 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.996876  1782 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:44.997522  1782 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:44.999467  1749 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:45.001535  1782 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:45.003070  1787 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:44.922941  1783 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:12:44.923055   988 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:44.923563 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:44.962077  1623 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.927068 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:45.006327 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 79359.6us
I20260709 13:12:45.007066 29122 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:45.008216  1580 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:12:44.946429  1747 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.551s	user 0.015s	sys 0.019s
I20260709 13:12:45.008680   988 raft_consensus.cc:2804] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:45.009007  1580 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:45.009320   988 raft_consensus.cc:697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:45.009840 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2600) took 2016 ms. Trace:
I20260709 13:12:45.009936  1580 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.010322   988 consensus_queue.cc:237] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.010295 29010 rpcz_store.cc:276] 0709 13:12:42.993324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.003749 (+ 10425us) service_pool.cc:224] Handling call
0709 13:12:43.043277 (+ 39528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:43.043625 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.043651 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:43.048340 (+  4689us) log.cc:844] Serialized 26 byte log entry
0709 13:12:43.254201 (+205861us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:43.254242 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.254317 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:44.945812 (+1691495us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:45.009568 (+ 63756us) raft_consensus.cc:1685] finished
0709 13:12:45.009588 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.009782 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":505,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":4879,"spinlock_wait_cycles":8662272,"thread_start_us":105377,"threads_started":1,"wal-append.queue_time_us":258575}
I20260709 13:12:44.953972  1615 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:45.012372  1706 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:45.008234 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:44.954689  1740 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:44.965140  1791 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.015228  1740 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:45.015672  1740 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:45.016070  1740 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:44.955905  1784 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:44.946154  1244 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:45.008260 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2676) took 1674 ms. Trace:
I20260709 13:12:44.956507  1761 consensus_queue.cc:237] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:45.096378 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.075187 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:45.045006 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.153424 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.017133 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:45.154706 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:45.154840 29083 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:45.023327 29007 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:45.157629 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:45.158427 29085 raft_consensus.cc:2393] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 14: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
W20260709 13:12:45.092278 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 258761us
I20260709 13:12:45.160969 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:45.161176 29680 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:45.158067 29156 rpcz_store.cc:276] 0709 13:12:43.333478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.333598 (+   120us) service_pool.cc:224] Handling call
0709 13:12:43.342500 (+  8902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.049684 (+707184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.008026 (+958342us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b54001cfcd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:45.008203 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1301242112}
I20260709 13:12:45.166348 29082 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:45.167256 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:45.167809 29082 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:45.168942 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:45.169970 29082 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:45.170130 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:45.006583  1781 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
I20260709 13:12:45.170714 29085 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:45.173854 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:45.174641 29085 raft_consensus.cc:2410] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:12:45.175590 29085 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:45.176282 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 280061us
W20260709 13:12:45.176810 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.964393  1785 raft_consensus.cc:3055] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:45.179160 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:45.145704  1791 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:45.145694  1795 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:45.187703 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 112573us
I20260709 13:12:45.185986 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 928) took 2236 ms. Trace:
I20260709 13:12:45.193010 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:45.182914  1785 raft_consensus.cc:740] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:45.179080  1782 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.190683 29082 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:12:45.197993 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:45.198024 29077 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:45.176940  1794 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.193043 29711 rpcz_store.cc:276] 0709 13:12:42.949797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.955521 (+  5724us) service_pool.cc:224] Handling call
0709 13:12:42.955783 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.955977 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:43.049668 (+ 93691us) log.cc:844] Serialized 14 byte log entry
0709 13:12:44.626396 (+1576728us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:45.148411 (+522015us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b18002ed500. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f41192393c7 00007f411584b9a7 00007f411585b0bf 00007f411585b038
0709 13:12:45.161304 (+ 12893us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.185666 (+ 24362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.185711 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:45.185762 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.185841 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.185938 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1908677,"prepare.queue_time_us":2219579,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":30129,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":11834,"spinlock_wait_cycles":13747328,"thread_start_us":273060,"threads_started":2,"wal-append.queue_time_us":292559}
I20260709 13:12:45.183911  1798 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:45.226492 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:45.195075  1726 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:45.182304  1690 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 13:12:45.206519 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.239598 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.229401 29079 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
W20260709 13:12:45.205621 29048 leader_election.cc:336] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.597s (SENT)
I20260709 13:12:45.237699 29077 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:45.242352 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:45.194625  1704 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:12:45.204231  1799 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:45.255764 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:45.257402 29077 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:45.258702  1386 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:45.222183  1794 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.261950  1794 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:45.249677  1669 consensus_meta.cc:287] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 13:12:45.220346  1782 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:45.276232 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 70641.9us
W20260709 13:12:45.304112 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.304561 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.248718  1492 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:45.298053 29077 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:45.250624  1615 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:45.311753 29715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47384 (request call id 934) took 2281 ms (client timeout 1694 ms). Trace:
W20260709 13:12:45.312372 29715 rpcz_store.cc:269] 0709 13:12:43.030671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.032390 (+  1719us) service_pool.cc:224] Handling call
0709 13:12:45.311499 (+2279109us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400650254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:45.311653 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504634112}
I20260709 13:12:45.240607  1784 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:12:45.341820 29082 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:12:45.392290 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51376.6us
I20260709 13:12:45.389612 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:45.238534  1798 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:45.398825 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:45.401093 29050 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:45.278252  1669 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:45.410270 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:45.412498 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:45.408968 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 183868us
I20260709 13:12:45.253491  1785 consensus_queue.cc:260] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.392608  1794 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:45.416796 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.420059 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:45.420856 29007 raft_consensus.cc:2393] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 15: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:45.417829 29006 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:45.422151 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:45.422967 29006 raft_consensus.cc:2368] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15.
I20260709 13:12:45.393375  1615 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:12:45.404659  1386 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:45.433148 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 235702us
I20260709 13:12:45.432451 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.434828 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:45.431217 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.436604 29006 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:45.410070  1802 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:45.437145 29680 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:45.437968 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.442266 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.439297  1386 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:45.419960  1616 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:45.414274  1791 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:45.442329 29007 raft_consensus.cc:2368] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14.
I20260709 13:12:45.455456 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.432825  1808 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:45.458426 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.464999 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.471073 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:45.470597 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.472205 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:45.473019 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.441332  1802 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:45.453580  1803 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:45.431231  1794 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.456961  1791 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:45.497213 29007 raft_consensus.cc:1240] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:12:45.484987  1802 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:45.538769  1807 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:12:45.498299  1791 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:45.572405 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:45.573794 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:45.577126  1818 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.578209  1790 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:45.579279 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.582422 29715 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:45.589291 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:45.475837  1808 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:45.480374  1740 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.521010  1716 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:45.605067 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.608592 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.617985 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:45.624652 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.616976 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:45.628130  1828 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:45.640287  1828 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:45.640709  1828 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:45.641412  1726 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:45.641960  1726 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:45.598973  1818 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:45.652791  1811 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:12:45.655562  1740 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:45.656644  1518 consensus_meta.cc:287] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.684s	user 0.006s	sys 0.006s
I20260709 13:12:45.657152  1518 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.658658  1615 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:45.659209  1615 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.628890  1716 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:45.662108  1615 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:45.662132  1716 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:45.597489  1759 consensus_meta.cc:287] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.628s	user 0.015s	sys 0.000s
W20260709 13:12:45.716357 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.720440 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.722576 29009 raft_consensus.cc:3055] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:45.722992 29009 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:45.692945  1820 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:12:45.692389  1759 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.736666  1802 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:45.754580 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.714640  1669 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:45.759487 29007 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:45.741185 29009 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:45.766728 29009 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:45.754375 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 334414us
I20260709 13:12:45.766752 29003 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:12:45.782932 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 123728us
I20260709 13:12:45.755126  1782 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:45.768813  1518 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:45.792357  1802 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:45.831432  1759 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:45.771723  1669 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:12:45.832724  1669 consensus_queue.cc:1243] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer 243d4ae4a4ae48b2979e32874472d256 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:45.833230  1669 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:45.833554  1669 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:45.834359  1669 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.835492  1669 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:45.773154  1794 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:45.891803 29009 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:12:45.892529 29009 pending_rounds.cc:85] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 7
I20260709 13:12:45.892870 29009 pending_rounds.cc:107] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:12:45.893224 29009 raft_consensus.cc:2889] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:45.894167 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2627) took 1120 ms. Trace:
I20260709 13:12:45.772645  1790 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 13:12:45.960316  1829 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:12:45.972901 29715 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:45.972718  1791 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.973219  1797 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:45.973771  1797 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:45.974944  1791 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:45.976667 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:45.976477  1726 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:45.976779 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:45.977459 29714 raft_consensus.cc:2410] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:12:45.977538 29708 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:45.978212  1738 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:45.978559  1726 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:45.978720  1738 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:45.980537 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.981148  1790 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:45.981276 29005 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:12:45.981712  1790 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:45.982751  1790 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:45.938478 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:45.941830 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:46.001292 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.894641 29009 rpcz_store.cc:276] 0709 13:12:44.773970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.774099 (+   129us) service_pool.cc:224] Handling call
0709 13:12:44.799147 (+ 25048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.722486 (+923339us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b54002687a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157b8dc0 00007f41158431c0 00007f4115844231 00007f4115840d8f 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c
0709 13:12:45.893844 (+171358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.894099 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25137024}
I20260709 13:12:45.937557 29712 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 13:12:46.009003 29712 pending_rounds.cc:85] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Aborting all ops after (but not including) 11
I20260709 13:12:46.009287 29712 pending_rounds.cc:107] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 13:12:46.009537 29712 raft_consensus.cc:2889] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:46.023617 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 88206.4us
I20260709 13:12:46.027735 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.029506 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:45.975891  1797 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:46.035842 29083 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:12:46.036108 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:46.037012 29082 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:46.039902 29080 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:46.057317 29712 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:46.047583 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:46.041353 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:46.066681 29080 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:46.067915 29080 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:46.069542 29081 raft_consensus.cc:2368] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14.
I20260709 13:12:46.025295  1790 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:46.071419 29009 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 13:12:46.073421 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:46.073011 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:46.075376 29083 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:46.028517  1791 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.081204 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:46.081817 29085 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:46.085418 29005 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:46.079218 29082 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:46.079661 29078 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:12:46.087112 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.082412 29683 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:46.087018 29078 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:46.091066 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.076728 29080 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:46.092931 29080 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:46.091650 29708 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:46.095148 29051 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:46.098402 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:46.070396  1579 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
W20260709 13:12:46.130790 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.136806 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.100971 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.138689 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.138535 29085 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:46.137843 29076 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:12:46.141893  1734 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.142449 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:46.142609  1734 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:46.146234  1734 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:46.095139  1791 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:46.148147 29081 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:46.155467  1155 raft_consensus.cc:3055] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:46.155932  1155 raft_consensus.cc:740] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:46.157981  1740 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:46.158279 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2635) took 1140 ms. Trace:
I20260709 13:12:46.159482  1738 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:46.158630 29010 rpcz_store.cc:276] 0709 13:12:45.018052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.018190 (+   138us) service_pool.cc:224] Handling call
0709 13:12:45.139001 (+120811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.161765 (+ 22764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.161811 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:45.176533 (+ 14722us) log.cc:844] Serialized 26 byte log entry
0709 13:12:45.244729 (+ 68196us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:45.244784 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.244870 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.153543 (+908673us) spinlock_profiling.cc:243] Waited 77.8 ms on lock 0x7b240012f038. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f41125da9c4 00007f41125d8acb 00007f4112793dc9 00007f4112793a55 00007f4112792fb0 00007f4115409488 00007f4115409be8 00007f41153e1398 00007f41153dc76a 00007f41153dbbb6
0709 13:12:46.157992 (+  4449us) raft_consensus.cc:1685] finished
0709 13:12:46.158013 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.158223 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25739,"raft.queue_time_us":44253,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":8349,"spinlock_wait_cycles":178905344,"thread_start_us":59617,"threads_started":1,"wal-append.queue_time_us":218830}
W20260709 13:12:46.160780 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.159114  1791 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:46.161373 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.097611 29078 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:46.162245 29078 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:46.164237 29005 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
W20260709 13:12:46.166781 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 123267us
I20260709 13:12:46.168651 29081 raft_consensus.cc:2410] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:12:46.032701  1828 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:46.190874 29005 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:12:46.188149 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.194525 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:46.195290 29005 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:46.196528 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:46.197081 29010 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:46.197777 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.096570  1830 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:46.098469  1716 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:46.099493  1802 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:46.126397  1757 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:12:46.205477 29009 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:12:46.222688 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.224004 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:46.224840 29009 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:46.208650 29080 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:46.211143 29076 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:46.221138 29078 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:46.204326  1765 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:46.237941 29001 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:46.213721  1716 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:46.242540 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 153766us
I20260709 13:12:46.225037  1798 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:46.259786 29009 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:46.227855  1802 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:46.235325   988 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:12:46.277773 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.278754 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.280341 29076 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:46.281847 29009 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:46.282913 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.205718  1155 consensus_queue.cc:260] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:46.284987 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46297.7us
I20260709 13:12:46.242282  1765 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:46.289635 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.290413  1765 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:46.290728 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.243953  1716 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:46.222719  1830 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:46.291793 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:46.293198 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.298195 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.300552 29010 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
W20260709 13:12:46.301899 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.303645 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.309171 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.312261 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.292375  1802 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:46.314844  1840 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:46.307361  1155 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:46.342139  1830 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:46.307106  1797 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:12:46.348768 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2611) took 1640 ms. Trace:
I20260709 13:12:46.359303 29008 rpcz_store.cc:276] 0709 13:12:44.707879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.710032 (+  2153us) service_pool.cc:224] Handling call
0709 13:12:44.710288 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.864167 (+153879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:44.864202 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:44.864248 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:44.864301 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.342550 (+1478249us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540012f054. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:46.348600 (+  6050us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.348713 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353377664}
I20260709 13:12:46.359330 29005 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:46.360977 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:46.349450 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.361920 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.362975 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:46.362353 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 104757us
I20260709 13:12:46.356022 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.373558 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.374804 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
W20260709 13:12:46.353492  1826 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.748s	user 0.006s	sys 0.015s
I20260709 13:12:46.366586 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:46.379156 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.390479 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.391160 29008 raft_consensus.cc:2368] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15.
I20260709 13:12:46.420121 29010 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:46.421454 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.422704 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:46.440559 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 74052.7us
I20260709 13:12:46.441257 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:46.388708  1693 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:46.454988 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 900) took 2551 ms. Trace:
I20260709 13:12:46.389933  1738 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:46.462381 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.455375 29084 rpcz_store.cc:276] 0709 13:12:43.885386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.885517 (+   131us) service_pool.cc:224] Handling call
0709 13:12:43.885924 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.074963 (+189039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:44.074994 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:44.075036 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:44.075077 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.436176 (+2361099us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400402254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:46.436217 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.436328 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434255616}
I20260709 13:12:46.393657  1759 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.406824  1839 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:46.453924  1832 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:12:46.490645 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.493412 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:46.496204 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:46.496893 29711 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:46.503129 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.455533  1745 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:46.518121 29084 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:46.478960  1840 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:12:46.537566 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.477622  1828 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:46.484213  1826 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:12:46.546785 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.553161 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.553231 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 67003.7us
I20260709 13:12:46.554178 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:46.527823  1842 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:12:46.563745 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.567754 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.545864  1803 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:12:46.599124  1795 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:12:46.545749  1824 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:46.618117 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.480387  1693 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:46.630334 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.487744  1759 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:46.625038 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.494226  1738 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:46.494992  1839 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:46.644025  1693 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:46.665606 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.670783 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 113099us
I20260709 13:12:46.618722  1834 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:46.604764  1745 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:46.671572 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:46.561367  1797 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:12:46.641171  1824 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:46.711800 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:46.729313 29125 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:46.622529  1740 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.083s
W20260709 13:12:46.811131 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.812676 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.814940 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.817025 29711 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
W20260709 13:12:46.819792 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 90580.7us
I20260709 13:12:46.813459 29007 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:12:46.839525 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.840484 29007 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:46.840320 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:46.760974  1518 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:46.845820 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 198794us
I20260709 13:12:46.847229 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:46.848480 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.802628  1846 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:46.851917 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.811795  1733 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:46.855297 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 44574.7us
I20260709 13:12:46.801414  1834 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:46.811411  1765 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:46.863029 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:46.851411  1518 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:46.869839 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.008632 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.009840 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 168551us
I20260709 13:12:46.850893  1821 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:46.922462 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.020902 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.960099 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.981432 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.001976 29682 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.27.63.126:43467, user_credentials={real_user=slave}} blocked reactor thread for 137574us
I20260709 13:12:46.850032  1840 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:47.029176 29048 leader_election.cc:336] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.607s (SENT)
I20260709 13:12:46.841884  1123 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:12:47.039876 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:47.078699 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.856273  1623 consensus_queue.cc:1059] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.166s
I20260709 13:12:46.865168  1824 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:47.087239 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58109.2us
I20260709 13:12:47.013913  1518 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.006196  1846 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:47.037449  1771 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:47.150063 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 139485us
I20260709 13:12:47.088650  1795 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:47.167737 29048 leader_election.cc:336] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.694s (SENT)
I20260709 13:12:47.169045 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:47.196365  1863 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:47.204483  1839 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:47.206099  1759 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:47.207023  1840 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.216729  1824 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:47.218499 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.243749 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 2.507s (SENT) [suppressed 27 similar messages]
W20260709 13:12:47.248060 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 90256.7us
W20260709 13:12:47.236379 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 378628us
I20260709 13:12:47.253875 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:47.260820 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.261284 29085 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:47.262498 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:47.082501  1821 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.019558  1745 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:47.278532 29006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.001s	sys 0.005s
W20260709 13:12:47.287514 29006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.002s	sys 0.005s
I20260709 13:12:47.277581 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:47.277766 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 630379us
W20260709 13:12:47.303977 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.304961  1871 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.299968  1867 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:47.322340 29009 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
W20260709 13:12:47.319185 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.325037 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.278107   909 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.001s	sys 0.006s
W20260709 13:12:47.326704   909 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.002s	sys 0.006s
I20260709 13:12:47.328192 29156 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:47.328316  1579 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:47.328526 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.329150 29157 raft_consensus.cc:2468] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 8.
I20260709 13:12:47.328966  1579 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.329443 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.328213 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.331029 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.331236  1579 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:47.331635 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.332170 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.333462 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:47.334764 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 312406us
I20260709 13:12:47.336226 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.337231 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:47.337782 29150 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:12:47.273675  1795 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:47.341269 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 63808.1us
I20260709 13:12:47.341588 29158 raft_consensus.cc:2393] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 17: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:47.343744 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:47.343935 29051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.013s	sys 0.013s
W20260709 13:12:47.345090 29051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.013s	sys 0.013s
W20260709 13:12:47.347078 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.348807 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.349705 29154 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
W20260709 13:12:47.350097  1790 consensus_meta.cc:287] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 1.270s	user 0.007s	sys 0.003s
I20260709 13:12:47.351673  1863 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.352461  1863 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.353315 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:47.354763 29711 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:12:47.148983  1834 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.354764 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.358619 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.171814  1771 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.317723  1738 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:47.290562  1859 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:12:47.354072  1518 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.369614  1761 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.369416  1790 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:47.367020  1623 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:12:47.373868  1860 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:12:47.374231  1623 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:47.377118 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 123317us
I20260709 13:12:47.273448  1851 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 13:12:47.380169 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:47.389910 29156 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:47.387070 29150 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:47.344412  1864 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:47.354970  1759 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:47.504575  1821 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:47.401427 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.560750 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.432233  1851 consensus_queue.cc:1243] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer ca45420e033847489ad989c1c6eed3ef log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:12:47.491665  1716 consensus_meta.cc:287] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 1.182s	user 0.002s	sys 0.006s
I20260709 13:12:47.515560  1729 raft_consensus.cc:3055] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:47.355683  1795 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.428592 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:47.445216 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.619084 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.401136  1797 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:47.450450 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.429725 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:47.477088 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 88015.6us
I20260709 13:12:47.622233 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:47.493197 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 153425us
W20260709 13:12:47.623397 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.623749 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.624068 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.623889 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.427768 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:47.401576  1864 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:47.420010 29680 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.538652 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.544446  1818 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.546944 29154 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:12:47.548759 29156 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:47.534664  1791 raft_consensus.cc:3055] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:47.650239  1791 raft_consensus.cc:740] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:47.651270  1795 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.651180  1791 consensus_queue.cc:260] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.535900 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 917) took 2113 ms. Trace:
I20260709 13:12:47.657770 29079 rpcz_store.cc:276] 0709 13:12:45.422105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.423613 (+  1508us) service_pool.cc:224] Handling call
0709 13:12:45.423921 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.439963 (+ 16042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.440010 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.440107 (+    97us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:45.440156 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.535609 (+2095453us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b540034f154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:47.535660 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:47.535839 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36309376}
I20260709 13:12:47.478928 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:47.659211 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 180416us
I20260709 13:12:47.661029 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:47.506044 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2668) took 2026 ms. Trace:
I20260709 13:12:47.662094 29006 rpcz_store.cc:276] 0709 13:12:45.481096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.512532 (+ 31436us) service_pool.cc:224] Handling call
0709 13:12:46.435022 (+922490us) spinlock_profiling.cc:243] Waited 6.35 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:46.435209 (+   187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:46.435553 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:46.435579 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:46.436000 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 13:12:47.288027 (+852027us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:47.321235 (+ 33208us) log.cc:844] Serialized 14 byte log entry
0709 13:12:47.321422 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.321493 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:47.505667 (+184174us) raft_consensus.cc:1685] finished
0709 13:12:47.505811 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:47.505973 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46040,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":30300,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":23955,"spinlock_wait_cycles":14605184,"thread_start_us":769350,"threads_started":1,"wal-append.queue_time_us":879984}
I20260709 13:12:47.664585 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:47.665277 29006 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:47.666594  1814 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:47.516810  1771 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:47.490882  1796 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.668028  1864 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:47.667867  1796 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.466526 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.375805  1761 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.673059  1761 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:47.410465 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.552024  1865 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:12:47.582180 29082 raft_consensus.cc:1240] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:47.583098  1518 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:47.593122  1785 raft_consensus.cc:3060] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:47.615867 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2643) took 2114 ms. Trace:
I20260709 13:12:47.572261  1869 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.678267 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.647105 29150 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:47.678855 29082 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 17.
W20260709 13:12:47.648365 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.618932  1831 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:47.652174  1821 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:47.681587 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.682382 29158 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:47.683725 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.685317 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.686126 29152 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:47.680754 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:47.652590  1791 raft_consensus.cc:3060] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:47.619150  1716 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.653645  1797 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:47.655025 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.696391 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.696902 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.697844 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.655757  1728 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:47.656446 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.655035 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.699707 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:47.603497  1729 raft_consensus.cc:740] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:47.375337  1623 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.603830  1740 consensus_queue.cc:1059] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:12:47.721632 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.682114  1616 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.677340 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.676378 29002 rpcz_store.cc:276] 0709 13:12:45.501677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.501820 (+   143us) service_pool.cc:224] Handling call
0709 13:12:46.635434 (+1133614us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ceee 0000000000375939 00007f411a7901e0 00007f411a8590c4 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:46.635623 (+   189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:46.635882 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:46.704819 (+ 68937us) log.cc:844] Serialized 14 byte log entry
0709 13:12:47.615021 (+910202us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b18003760c0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a
0709 13:12:47.615356 (+   335us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:47.615441 (+    85us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:47.615511 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.615611 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:47.615753 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28582,"prepare.queue_time_us":171849,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":5176,"raft.queue_time_us":5148,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":27733,"spinlock_wait_cycles":609600000,"thread_start_us":149891,"threads_started":1,"wal-append.queue_time_us":603726}
I20260709 13:12:47.716830 28976 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:47.690017 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:47.693002 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2663) took 1102 ms. Trace:
I20260709 13:12:47.693835 29150 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:12:47.727319 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:47.726493 29003 rpcz_store.cc:276] 0709 13:12:46.591149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.858639 (+267490us) service_pool.cc:224] Handling call
0709 13:12:46.859625 (+   986us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:46.859946 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:46.859976 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:46.860435 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:12:47.051035 (+190600us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:47.051069 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.051140 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:47.539504 (+488364us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:47.677540 (+138036us) raft_consensus.cc:1685] finished
0709 13:12:47.677560 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:47.692948 (+ 15388us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":392407,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":28182,"spinlock_wait_cycles":7492864,"thread_start_us":190292,"threads_started":1,"wal-append.queue_time_us":441830}
I20260709 13:12:47.704622 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:47.687907  1492 consensus_queue.cc:1059] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:12:47.715484 29151 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: []
W20260709 13:12:47.716751 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.729110 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2666) took 1075 ms. Trace:
I20260709 13:12:47.729317  1151 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:47.670473  1796 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:47.666651  1745 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:47.642374 29125 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:47.730811 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 88527.6us
I20260709 13:12:47.731482  1759 raft_consensus.cc:2804] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:47.731917 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:47.677004 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.733510  1693 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:47.733383  1851 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:47.736071  1851 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:47.736450  1851 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:47.732012  1759 raft_consensus.cc:697] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:47.740993  1759 consensus_queue.cc:237] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.743584  1797 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.733371 29125 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:47.745776 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.746428  1745 raft_consensus.cc:2804] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:47.746956  1745 raft_consensus.cc:697] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:47.729563 29002 rpcz_store.cc:276] 0709 13:12:46.653937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.728346 (+1074409us) service_pool.cc:224] Handling call
0709 13:12:47.728608 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:47.728779 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:47.728807 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:47.728851 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:47.728907 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.728961 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:47.729058 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:47.747841  1745 consensus_queue.cc:237] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.726176 29078 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:47.757611  1716 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.692802  1831 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:47.759071 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.692720  1821 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.725035 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:47.761557  1821 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.763630  1729 consensus_queue.cc:260] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.765110  1729 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:47.770582  1771 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.729784  1151 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:47.771158  1771 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.773293  1771 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.771435  1151 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.780102 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.780825 29151 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:47.782071 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.714156  1728 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.786713  1728 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.786633 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:47.700971  1795 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.787508  1795 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.788317 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:47.788833 29078 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:47.789728  1795 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.790382  1782 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:47.688180  1869 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.793607  1869 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:47.794462  1857 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:12:47.796634 29152 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:47.798693 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 65384.8us
I20260709 13:12:47.798776 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.693907  1761 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:47.801339 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.794781  1728 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:47.801628 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:47.818419 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 101674us
I20260709 13:12:47.802111 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:45.682544 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:47.823848 29159 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:47.828828 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:47.775826  1864 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.830277  1738 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:47.831879  1864 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:47.831996 29125 leader_election.cc:400] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14.
I20260709 13:12:47.800825  1616 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.832501 29125 leader_election.cc:403] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:47.833039  1864 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14.
I20260709 13:12:47.834311  1616 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:47.809881  1795 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.834893  1864 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:47.835064  1795 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:47.836404 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:47.837117  1864 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:47.837072  1795 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.835908  1151 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:47.810056  1728 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.840345  1728 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:47.840643  1151 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:47.842487 29078 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
I20260709 13:12:47.844344  1799 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:47.844182 29683 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:47.846232 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:47.846772 29717 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:47.847723 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:47.847961  1803 raft_consensus.cc:2804] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:47.848428 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:47.849745 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:47.846267 28976 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:47.846248 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:47.807659  1761 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.852986 29155 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:12:47.850741 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:47.848502  1803 raft_consensus.cc:697] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:47.818915  1782 raft_consensus.cc:2764] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:12:47.870669 29076 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:47.871184 29082 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:47.875142 29082 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:47.876907 29713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 947) took 2304 ms. Trace:
I20260709 13:12:47.877285 29713 rpcz_store.cc:276] 0709 13:12:45.572651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.572774 (+   123us) service_pool.cc:224] Handling call
0709 13:12:45.576946 (+  4172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.660896 (+ 83950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.660927 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.660969 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:12:45.661000 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.876649 (+2215649us) spinlock_profiling.cc:243] Waited 83.8 ms on lock 0x7b5400502154. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:47.876710 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:47.876846 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192635648}
I20260709 13:12:47.822853  1118 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:47.884203 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.885259 29081 raft_consensus.cc:2393] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:47.868255 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.865549 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:47.886597 29006 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.882987 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.854169 28976 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:47.888975 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.877415 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2664) took 1257 ms. Trace:
I20260709 13:12:47.889742 29081 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 17: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:47.890902 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.891655 29084 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:47.893468 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.894132 29084 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:47.890050 29008 rpcz_store.cc:276] 0709 13:12:46.620340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.312898 (+692558us) service_pool.cc:224] Handling call
0709 13:12:47.313253 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:47.313542 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:47.363020 (+ 49478us) log.cc:844] Serialized 14 byte log entry
0709 13:12:47.596641 (+233621us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b1800388c60. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:12:47.877108 (+280467us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:47.877156 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:47.877206 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.877266 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:47.877366 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21011,"prepare.queue_time_us":48661,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":387,"spinlock_wait_cycles":11623680,"thread_start_us":279616,"threads_started":1}
I20260709 13:12:47.851552  1793 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:47.808193  1721 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:12:47.861546  1761 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:47.903615 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:47.916137 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.911325 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:47.844226  1821 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:12:47.911978 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 57875.9us
I20260709 13:12:47.920464 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:47.921193 29712 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.825784  1846 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:12:47.896319  1814 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:47.966441  1814 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:47.959842 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.964362 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:47.965260 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:47.970072 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:47.970438 29085 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:47.971755 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.974728 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:47.977067 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:47.982605 29076 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:12:47.996903 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:48.000929 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:48.001705 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.002619 29006 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:47.999200 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:48.004585 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.012380 29085 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:48.021014  1808 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:47.995497  1840 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:12:48.023388 29076 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
W20260709 13:12:48.024309  1887 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 13:12:48.024941  1881 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:12:48.025146  1728 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:48.026086 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.084627 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.034462 29010 raft_consensus.cc:2410] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:12:48.031725  1851 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.087352  1851 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:48.034700  1790 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.088230  1790 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:47.961989  1782 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:12:48.054951  1760 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:48.054841 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:48.016798 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.095211 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.016018   988 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:48.078346 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2665) took 1454 ms. Trace:
I20260709 13:12:48.015323  1793 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:48.090531 29159 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:48.013063  1845 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:12:47.986737  1814 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:48.098835 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 44126.4us
I20260709 13:12:48.116644 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:48.097565 29003 rpcz_store.cc:276] 0709 13:12:46.624483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.728064 (+1103581us) service_pool.cc:224] Handling call
0709 13:12:47.728320 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:47.800622 (+ 72302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:47.800650 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:47.800696 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:47.800728 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.078095 (+277367us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b54001d74d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:48.078142 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:48.078287 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165876992}
I20260709 13:12:48.100136 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2667) took 1434 ms. Trace:
W20260709 13:12:48.103277 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.105360 29006 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:48.120024 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 908) took 1260 ms. Trace:
I20260709 13:12:48.120357 29006 rpcz_store.cc:276] 0709 13:12:46.859766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.104863 (+1245097us) service_pool.cc:224] Handling call
0709 13:12:48.105172 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:48.119781 (+ 14609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.119973 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:48.120528 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:48.090541 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 120559us
I20260709 13:12:47.984587  1803 consensus_queue.cc:237] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:48.114864 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.131153 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:48.105237   988 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.117550 29002 rpcz_store.cc:276] 0709 13:12:46.665453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.749071 (+1083618us) service_pool.cc:224] Handling call
0709 13:12:47.749394 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:47.749628 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:47.750517 (+   889us) log.cc:844] Serialized 14 byte log entry
0709 13:12:47.925908 (+175391us) log.cc:844] Serialized 14 byte log entry
0709 13:12:48.099639 (+173731us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:48.099714 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:48.099787 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.099882 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:48.100033 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38984,"prepare.queue_time_us":1667,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":65,"raft.queue_time_us":58303,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":19776,"thread_start_us":308527,"threads_started":2,"wal-append.queue_time_us":337689}
I20260709 13:12:48.098075  1760 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:48.128271 29712 raft_consensus.cc:2468] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:48.106943   909 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:48.080579  1118 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:12:48.122747 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:48.162624  1846 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:48.168917 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.192288 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:48.196693 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 52862.6us
I20260709 13:12:48.201015 29150 raft_consensus.cc:1240] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:12:48.202386 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:48.203889 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.203899 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.207985 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.208325 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.198886 29125 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:48.209338 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.154106   988 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:48.212756 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.202970 29152 raft_consensus.cc:2376] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14. Re-sending same reply.
I20260709 13:12:48.213773 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.215940 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:48.220140 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:48.220265 29682 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:48.220983 29150 raft_consensus.cc:2376] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13. Re-sending same reply.
I20260709 13:12:48.221855 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:48.222066 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.161129  1795 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:48.133773  1851 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
W20260709 13:12:48.228101 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 105412us
I20260709 13:12:48.163913  1760 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:48.224888 29682 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:48.225296 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.225548 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:48.230048 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.231047 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:48.210651  1791 consensus_meta.cc:287] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.517s	user 0.002s	sys 0.007s
I20260709 13:12:48.231213 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.155102  1818 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:12:48.163385  1814 consensus_queue.cc:1059] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:12:48.240545 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.228816 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:48.189550  1821 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:12:48.168226  1793 consensus_queue.cc:1059] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:12:48.225418  1492 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:48.226130  1859 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:48.226016  1876 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:48.223973  1808 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:48.225188  1795 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.241102  1759 raft_consensus.cc:2804] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:48.311021 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 154225us
I20260709 13:12:48.319458 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.244344  1704 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:48.253351  1898 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:12:48.287726 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 62891.5us
W20260709 13:12:48.395665 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.288599  1492 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:48.403159 29682 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:48.299386  1876 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:48.402094 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 249937us
I20260709 13:12:48.412144 29050 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:48.318025  1795 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:48.375380  1885 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:12:48.265431  1799 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.149s
W20260709 13:12:48.413151 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 184396us
I20260709 13:12:48.382336  1759 raft_consensus.cc:697] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:48.455549 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.368075  1793 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:48.459425 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:48.463325 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:48.454746 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51655.8us
I20260709 13:12:48.382635  1728 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:48.402033  1898 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:48.414357  1492 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:48.469780 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:48.414575  1876 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:48.477378 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:48.485657  1895 log.cc:927] Time spent T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
W20260709 13:12:48.486980  1904 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 13:12:48.488034  1901 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:12:48.499372  1729 consensus_meta.cc:287] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.734s	user 0.009s	sys 0.002s
I20260709 13:12:48.500015 29715 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:48.464905  1864 consensus_queue.cc:1059] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 13:12:48.506790  1849 log.cc:927] Time spent T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.980s	user 0.007s	sys 0.008s
I20260709 13:12:48.462865  1836 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:48.505215  1863 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:48.514521  1899 log.cc:927] Time spent T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:12:48.515969  1821 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:48.516547  1821 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.518798  1821 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:48.519554  1793 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:48.524545  1759 consensus_queue.cc:237] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.474815  1898 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:48.508679  1693 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:48.509454  1728 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:48.562831 29082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.004s	sys 0.013s
I20260709 13:12:48.548506  1915 raft_consensus.cc:3055] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:12:48.563936 29082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.004s	sys 0.013s
W20260709 13:12:48.569541 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 106279us
I20260709 13:12:48.583339 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:48.562287  1693 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:48.586031 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:48.587458  1916 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:48.585798  1814 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:12:48.597177  1761 consensus_meta.cc:287] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.000s
I20260709 13:12:48.597733  1761 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.599145 29716 raft_consensus.cc:2368] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 17. Current term is 18.
I20260709 13:12:48.603435  1727 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:48.565737  1915 raft_consensus.cc:740] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:48.614500 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:48.615006 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.616006 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:48.616575 29081 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:12:48.621447   988 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.622890 29081 raft_consensus.cc:2468] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:12:48.623513   988 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:48.624609  1693 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.625597 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:48.626562 29683 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:48.627355  1836 raft_consensus.cc:2804] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:48.612597  1799 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:12:48.629171  1915 consensus_queue.cc:260] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.630472  1915 raft_consensus.cc:3060] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:48.631453  1868 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.265s	user 0.006s	sys 0.000s
I20260709 13:12:48.633291 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2662) took 2068 ms. Trace:
I20260709 13:12:48.633630 29007 rpcz_store.cc:276] 0709 13:12:46.564455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.858242 (+293787us) service_pool.cc:224] Handling call
0709 13:12:46.863354 (+  5112us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:47.094758 (+231404us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:47.146946 (+ 52188us) log.cc:844] Serialized 14 byte log entry
0709 13:12:47.366364 (+219418us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:47.388997 (+ 22633us) log.cc:844] Serialized 52 byte log entry
0709 13:12:47.389156 (+   159us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:47.389192 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.389257 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:48.630895 (+1241638us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:48.633022 (+  2127us) raft_consensus.cc:1685] finished
0709 13:12:48.633112 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:48.633236 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158418,"prepare.queue_time_us":44505,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":427395,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":23841,"spinlock_wait_cycles":15558016,"thread_start_us":270749,"threads_started":2,"wal-append.queue_time_us":192270}
I20260709 13:12:48.627832  1836 raft_consensus.cc:697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:48.636271  1836 consensus_queue.cc:237] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:48.647361 29010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2692) took 2241 ms (client timeout 2571 ms). Trace:
W20260709 13:12:48.647790 29010 rpcz_store.cc:269] 0709 13:12:46.405899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.422463 (+ 16564us) service_pool.cc:224] Handling call
0709 13:12:48.647164 (+2224701us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400291b54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:48.647282 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3704564608}
I20260709 13:12:48.649557  1761 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:48.650672 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:48.648401 29154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2708) took 1257 ms. Trace:
I20260709 13:12:48.653183 29154 rpcz_store.cc:276] 0709 13:12:47.391020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.391190 (+   170us) service_pool.cc:224] Handling call
0709 13:12:47.400472 (+  9282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:47.649631 (+249159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.648203 (+998572us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b54002b52d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:48.648341 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12081920}
I20260709 13:12:48.656013 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.656780 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.658046 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:48.658298  1819 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:48.658736  1819 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:48.659780  1761 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:48.660610 29003 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:48.662212  1492 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.664325  1791 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.664867  1791 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.667591  1791 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:48.668644  1846 raft_consensus.cc:3055] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:48.669049  1846 raft_consensus.cc:740] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:48.669847  1846 consensus_queue.cc:260] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.671078  1846 raft_consensus.cc:3060] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:48.657824  1876 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.675081  1821 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:48.677961  1906 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:48.679307 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.681531  1760 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.659581  1819 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.672231  1790 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:48.684607  1883 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.077s	user 0.002s	sys 0.003s
I20260709 13:12:48.705073 29158 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:48.705837  1819 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:48.706331  1760 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:48.687820 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:48.709154  1920 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.709795  1798 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:12:48.709924 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 93793.2us
I20260709 13:12:48.710350  1798 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:48.711282  1798 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.712330 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2644) took 3187 ms. Trace:
I20260709 13:12:48.712715 29004 rpcz_store.cc:276] 0709 13:12:45.525510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.576494 (+ 50984us) service_pool.cc:224] Handling call
0709 13:12:47.545060 (+1968566us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:47.545271 (+   211us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:47.545679 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:47.545710 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:47.546346 (+   636us) log.cc:844] Serialized 52 byte log entry
0709 13:12:47.558758 (+ 12412us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:47.558813 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.558879 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:48.683892 (+1125013us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:48.712106 (+ 28214us) raft_consensus.cc:1685] finished
0709 13:12:48.712129 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:48.712272 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6032,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6014,"spinlock_wait_cycles":41387008,"thread_start_us":12009,"threads_started":1,"wal-append.queue_time_us":53601}
I20260709 13:12:48.713781 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.623698  1795 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.715819  1795 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:48.715687  1304 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:48.716514  1784 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:48.716780  1920 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.693501  1821 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:48.718948 29084 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 18: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:48.719213  1821 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.720314  1920 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:48.720649 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:48.720829 29156 raft_consensus.cc:3055] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:48.721189 29156 raft_consensus.cc:740] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:48.723431 29713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "027a3f0877db4e259d94a8e296822644"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:51170
I20260709 13:12:48.724315 29713 raft_consensus.cc:606] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:48.724758 29713 raft_consensus.cc:614] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:12:48.718142  1761 consensus_queue.cc:1175] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "06ae4197fd784525b695a9b16c8094ea" responder_term: 12 status { last_received { term: 11 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:48.697145 29710 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:48.721822 29156 consensus_queue.cc:260] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.690213  1898 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.727624 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.728142 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:48.734766 29085 raft_consensus.cc:2368] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:48.728384 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.729213 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:48.730216 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.735780 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:48.730677 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:48.731271 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.736812 29082 raft_consensus.cc:2368] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:48.737178 29083 raft_consensus.cc:2368] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:48.744306 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:48.744364 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.745968 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.746390 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.747365 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.748000 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.748867 29085 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:48.749894 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.750950 29085 raft_consensus.cc:2410] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:12:48.753994 29710 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 14. Ops: [12.10-12.10]
I20260709 13:12:48.755618 29710 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:48.756206 29708 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:48.756377 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:48.757948 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:48.759380 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:48.760716 29150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31c5d53651b5426780911903843f0bc3"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:48.761339 29150 raft_consensus.cc:606] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:48.765784  1876 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:48.768306 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:48.771167  1854 log.cc:927] Time spent T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 1.675s	user 0.002s	sys 0.010s
I20260709 13:12:48.740267  1906 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:48.774137 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:48.774317 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 362208us
I20260709 13:12:48.774771 29150 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:48.779120 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:48.779453 29050 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:48.780784 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:48.785730  1906 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:48.786083 29717 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 15. Ops: []
I20260709 13:12:48.790544  1831 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:48.792057  1920 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:48.787084  1836 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.793180  1836 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.793797 29150 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
W20260709 13:12:48.795099 29009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33750 (request call id 906) took 2342 ms (client timeout 1850 ms). Trace:
W20260709 13:12:48.795490 29009 rpcz_store.cc:269] 0709 13:12:46.452568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.839927 (+387359us) service_pool.cc:224] Handling call
0709 13:12:48.794907 (+1954980us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54003f0cd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:48.795030 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099848832}
I20260709 13:12:48.799302 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2675) took 1446 ms. Trace:
I20260709 13:12:48.803821 29085 raft_consensus.cc:1240] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:48.804419 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.806562  1790 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:48.805100 29003 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:48.811780 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:48.813002  1912 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:48.813378 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:48.814109 29151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a08e186ea0749ae979329103591a933"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:48.821090 29151 raft_consensus.cc:606] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:48.815214 29717 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:12:48.817452 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.799690 29002 rpcz_store.cc:276] 0709 13:12:47.352665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.156544 (+803879us) service_pool.cc:224] Handling call
0709 13:12:48.797473 (+640929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:48.797653 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:48.797676 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:48.797715 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:48.797744 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.797795 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:48.799110 (+  1315us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54003f0cd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:48.799243 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472621952}
I20260709 13:12:48.829998 29717 raft_consensus.cc:1240] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:48.795787  1836 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:48.834262 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 48728.3us
I20260709 13:12:48.836016 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:48.831951  1920 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:12:48.839089 29085 raft_consensus.cc:1275] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:12:48.839452 29085 pending_rounds.cc:85] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 7
I20260709 13:12:48.839654 29085 pending_rounds.cc:107] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:12:48.839865 29085 raft_consensus.cc:2889] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:48.840830 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:48.841336 29085 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:48.841979 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:48.843544 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:48.842947 29682 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:48.848313 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.849074 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:48.848951 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.852226 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.820191  1912 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.854305 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2657) took 2488 ms. Trace:
I20260709 13:12:48.867328 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:48.868224 29717 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 11.
I20260709 13:12:48.868567 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:48.848227  1567 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:48.875753 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.877046 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.882077 29003 raft_consensus.cc:2368] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15.
I20260709 13:12:48.885841 29712 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:48.886242 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.887315 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.888320 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.889020 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.889253 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.890213 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.890476 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.891209 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:48.860687  1818 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:48.892474  1818 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:48.852437  1906 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:48.851855  1704 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:48.861994  1912 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:48.893327  1818 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.894024  1906 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:48.898051 29050 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:48.899431  1928 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:48.899945  1928 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:48.903295  1898 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:48.904452  1693 consensus_queue.cc:1048] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:48.893147 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.907716 29003 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:48.914070 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.916441 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.917207 29004 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.918776 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.921118 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.922775 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.923007 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2673) took 1571 ms. Trace:
I20260709 13:12:48.925196 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.859988 29001 rpcz_store.cc:276] 0709 13:12:46.365731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.384265 (+ 18534us) service_pool.cc:224] Handling call
0709 13:12:46.384601 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:46.384972 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:46.385010 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:46.385557 (+   547us) log.cc:844] Serialized 52 byte log entry
0709 13:12:46.862602 (+477045us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:47.050083 (+187481us) log.cc:844] Serialized 14 byte log entry
0709 13:12:48.775435 (+1725352us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b1800335320. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:12:48.845122 (+ 69687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.845235 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:48.845323 (+    88us) raft_consensus.cc:1685] finished
0709 13:12:48.854073 (+  8750us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:48.854235 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33528,"prepare.queue_time_us":186107,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":6092,"raft.queue_time_us":256376,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":6513,"spinlock_wait_cycles":76659072,"thread_start_us":492612,"threads_started":2,"wal-append.queue_time_us":946003,"wal-append.run_cpu_time_us":2967,"wal-append.run_wall_time_us":1100116}
I20260709 13:12:48.935592 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.935725 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:48.936916 29001 raft_consensus.cc:2368] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
I20260709 13:12:48.937834 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.924335 29006 rpcz_store.cc:276] 0709 13:12:47.352061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.152719 (+800658us) service_pool.cc:224] Handling call
0709 13:12:48.153010 (+   291us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:48.153402 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:48.153429 (+    27us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:12:48.166889 (+ 13460us) log.cc:844] Serialized 78 byte log entry
0709 13:12:48.725311 (+558422us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b18004847e0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e10d1 00007f41157fca0f
0709 13:12:48.870358 (+145047us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:48.870404 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.870472 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:48.874367 (+  3895us) raft_consensus.cc:1685] finished
0709 13:12:48.874397 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:48.922926 (+ 48529us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6319,"raft.queue_time_us":1896,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":2381,"spinlock_wait_cycles":5501824,"thread_start_us":144842,"threads_started":1,"wal-append.queue_time_us":681191}
I20260709 13:12:48.939101 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.939296 29002 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 15: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:48.943857 29085 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:48.945168 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:48.945448 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:48.945855 29085 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:48.946507  1839 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:48.946967 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:48.947664 29085 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:48.948093 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.948362 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.949286 29006 raft_consensus.cc:2368] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
W20260709 13:12:48.949913 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.947230 29125 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:48.978232 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.956240 29004 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:48.922185  1623 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:48.976792 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:48.947026  1839 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:49.028812 29716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49414 (request call id 749) took 1182 ms. Trace:
I20260709 13:12:49.001551 29085 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:48.967847  1745 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:49.005172 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.006368 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.007274 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:49.033241 29084 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:49.007640 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:49.034153 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.009289 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.019837 29002 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:49.034879 29006 raft_consensus.cc:2410] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:12:49.035357 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.024603 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:49.036739 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:49.029208 29716 rpcz_store.cc:276] 0709 13:12:47.846848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.850510 (+  3662us) service_pool.cc:224] Handling call
0709 13:12:49.027446 (+1176936us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b540068ecd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:49.028747 (+  1301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702975872}
I20260709 13:12:49.029860 29080 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:12:49.027067 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:49.037653 29080 pending_rounds.cc:85] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 8
I20260709 13:12:49.037892 29080 pending_rounds.cc:107] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:12:49.038110 29080 raft_consensus.cc:2889] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:49.005079  1880 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.490s	user 0.005s	sys 0.000s
I20260709 13:12:49.040899 29009 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:12:49.041275 29009 pending_rounds.cc:85] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 8
I20260709 13:12:49.041481 29009 pending_rounds.cc:107] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:12:49.041694 29009 raft_consensus.cc:2889] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:49.042981 29009 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:12:49.035938 29002 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:49.041694 29003 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:49.030828 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.048182 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 2023) took 1270 ms. Trace:
I20260709 13:12:49.050582 29155 rpcz_store.cc:276] 0709 13:12:47.777825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.780158 (+  2333us) service_pool.cc:224] Handling call
0709 13:12:47.780460 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:47.861901 (+ 81441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:49.047879 (+1185978us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x7b540013e554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:49.048111 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166084224}
I20260709 13:12:49.051928 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.052112 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:49.054068 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:49.060949 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.041859  1745 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.061909 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.062111 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.062558 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.040163  1836 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:49.063834 29008 raft_consensus.cc:2368] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15.
I20260709 13:12:49.064146 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.066831 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:49.073140 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.073882 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.067970 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.076303 29003 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:49.064759 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.054805  1492 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:49.063082 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.060554  1928 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:49.090250 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.091271 29003 raft_consensus.cc:2410] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:12:49.091929   988 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:49.094394 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.095005 29006 raft_consensus.cc:2468] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
I20260709 13:12:49.098469 29084 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:49.068682  1745 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.106007 29006 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:49.107438 29006 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:49.092533  1151 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:49.118182  1151 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:49.119037  1151 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:49.120316 29156 raft_consensus.cc:3060] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.115734 29081 raft_consensus.cc:1275] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:12:49.066576  1795 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:49.129449 29081 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:49.129945 29156 raft_consensus.cc:1275] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:49.130492 29151 raft_consensus.cc:2468] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:49.130620 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 796) took 1477 ms. Trace:
I20260709 13:12:49.131063 29156 rpcz_store.cc:276] 0709 13:12:47.652830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.653071 (+   241us) service_pool.cc:224] Handling call
0709 13:12:47.653395 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:48.720744 (+1067349us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b54001cdf20. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157b8dc0 00007f41158431c0 00007f4115844231 00007f4115840d8f 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c
0709 13:12:49.119891 (+399147us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b58006dd6d0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f411579d0aa 00007f411581f447 00007f4115837fe7 00007f41158397de 00007f4115843b49 00007f4115844416 00007f4115840d8f 00007f411584083e 00007f411a805419 00007f4113d7cbd4
0709 13:12:49.130372 (+ 10481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:49.130570 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070589696}
I20260709 13:12:49.133121  1839 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:49.136734  1795 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:49.138870  1795 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:49.144296  1123 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:12:49.147217 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.148152 29003 raft_consensus.cc:2410] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:12:49.148917 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:49.150111 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:49.151697 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.157539 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:49.158509 29084 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:12:49.159435 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:49.165863 29005 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.548s	user 0.008s	sys 0.012s
W20260709 13:12:49.166235 29005 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.554s	user 0.009s	sys 0.013s
I20260709 13:12:49.166965 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2656) took 2801 ms. Trace:
I20260709 13:12:49.169480 29048 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.170745 29048 leader_election.cc:400] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15.
I20260709 13:12:49.171262 29125 leader_election.cc:400] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 17. Current term is 18.
I20260709 13:12:49.171547 29125 leader_election.cc:403] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:49.172122  1834 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:12:49.172307  1914 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:49.174364 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:49.174867 29714 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:49.177043  1123 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:49.182327  1123 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:49.183491  1123 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:49.180493  1792 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:49.180982  1897 raft_consensus.cc:3055] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:49.184963 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:49.186017 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:49.188858 29081 raft_consensus.cc:1240] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:12:49.191243 29083 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:12:49.191200 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.191824 29081 raft_consensus.cc:1240] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 14. Ops: [12.10-12.10]
I20260709 13:12:49.167309 29005 rpcz_store.cc:276] 0709 13:12:46.365418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.382099 (+ 16681us) service_pool.cc:224] Handling call
0709 13:12:46.394674 (+ 12575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:46.394897 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:12:46.611720 (+216823us) log.cc:844] Serialized 14 byte log entry
0709 13:12:49.004574 (+2392854us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41153dc86e 00007f41153dbbb6 00007f41153db7d8 00007f41153e5221 00007f41153e51d9 00007f41153e5169
0709 13:12:49.166607 (+162033us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:49.166665 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:12:49.166713 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:49.166793 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:49.166911 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":251411,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":751,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":104,"spinlock_wait_cycles":20621184,"thread_start_us":2726089,"threads_started":2,"wal-append.queue_time_us":902796}
I20260709 13:12:49.200712 29048 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.202129 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:49.202584 29083 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:49.206214  1518 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:49.208129  1908 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:49.203712  1771 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:49.209148  1771 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:49.209556  1771 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:49.189705  1897 raft_consensus.cc:740] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:49.210629  1897 consensus_queue.cc:260] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:49.211961  1897 raft_consensus.cc:3060] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.208520  1909 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:49.115796  1912 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:49.188119 29006 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:49.217350 29048 leader_election.cc:400] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
I20260709 13:12:49.217974 29048 leader_election.cc:403] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:12:49.217736 29001 raft_consensus.cc:3055] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:49.218461 29001 raft_consensus.cc:740] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:49.219480 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:49.222148 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.224599 29048 leader_election.cc:400] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
I20260709 13:12:49.219157 29001 consensus_queue.cc:260] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:49.225566 29006 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:12:49.227005 28976 leader_election.cc:400] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:49.225525 29001 raft_consensus.cc:3060] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:12:49.197247  1795 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.225528 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:49.241108 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.196807  1792 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:49.250105 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:49.250310 29076 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:49.255244  1771 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:49.256206  1738 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:49.260651 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.268632 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:49.261996 29006 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:49.263347 29709 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 14. Ops: []
I20260709 13:12:49.228574  1908 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:49.228808  1909 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 13.
I20260709 13:12:49.262658  1869 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:49.295692   988 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:49.270860 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:49.284837  1785 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 15 is higher.
I20260709 13:12:49.274689  1917 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:49.274377  1623 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:49.237516 28976 leader_election.cc:400] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:49.306449  1846 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:12:49.306268  1906 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:49.295405  1827 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:12:49.330617 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.331615 29001 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:12:49.331820 28976 leader_election.cc:403] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:12:49.341166 29001 pending_rounds.cc:85] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 9
I20260709 13:12:49.341557 29001 pending_rounds.cc:107] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:12:49.341836 29001 raft_consensus.cc:2889] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:49.344069 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 106644us
I20260709 13:12:49.345112 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:49.345444  1924 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:49.345572 29048 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.348122 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:49.352007 28976 leader_election.cc:400] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:49.352252 29048 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.330856  1917 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:49.355036 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:49.355950 29001 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:12:49.357378 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:49.344069  1834 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 17. Current term is 18.
I20260709 13:12:49.347164  1795 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:49.347743  1567 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:49.351194  1786 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:49.354614  1787 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:49.351002  1836 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:12:49.373129  1623 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:49.496029  1771 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:49.498091  1936 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:12:49.502924  1917 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:49.523128  1716 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:49.600772  1716 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:49.601836  1716 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:49.603447  1716 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:49.618567  1912 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:49.578083  1792 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:49.367357  1906 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:49.371071  1938 log.cc:927] Time spent T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 13:12:49.798301 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.800450 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.802445 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.803351 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.803719 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.804028 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.804448 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.804837 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.812408 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 460334us
I20260709 13:12:49.814179 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:49.818034 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:49.829715  1950 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:49.831902 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 474394us
I20260709 13:12:49.833659 29048 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.829751 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.837294 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2383) took 1002 ms. Trace:
I20260709 13:12:49.838459 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 950) took 2093 ms. Trace:
I20260709 13:12:49.838999 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:49.839318 29084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.007s	sys 0.012s
W20260709 13:12:49.839942 29084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.007s	sys 0.012s
I20260709 13:12:49.842968 29076 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:49.364912 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:49.845148 29083 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:12:49.826732 28904 catalog_manager.cc:5697] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 6 to 8, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 8 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:49.839956 29078 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 13:12:49.839046 29079 rpcz_store.cc:276] 0709 13:12:47.745055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.745536 (+   481us) service_pool.cc:224] Handling call
0709 13:12:49.836241 (+2090705us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:49.837088 (+   847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.837702 (+   614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:49.837747 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:49.837829 (+    82us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:49.837921 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:49.838017 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:49.838181 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420608}
I20260709 13:12:49.843498 29048 leader_election.cc:400] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15.
I20260709 13:12:49.854251 29048 leader_election.cc:403] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:49.851169 29083 pending_rounds.cc:85] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 7
I20260709 13:12:49.837877 29077 rpcz_store.cc:276] 0709 13:12:48.835183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.020552 (+185369us) service_pool.cc:224] Handling call
0709 13:12:49.020852 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:49.021234 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:49.021279 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:49.021768 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:12:49.091008 (+ 69240us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:49.091048 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:49.091118 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:49.836340 (+745222us) raft_consensus.cc:1685] finished
0709 13:12:49.836386 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:49.837049 (+   663us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1326,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":68977,"threads_started":1,"wal-append.queue_time_us":252948}
I20260709 13:12:49.872356  1787 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:49.872905 29081 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:12:49.873693  1792 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:49.873924  1787 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.872146  1785 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.874969  1787 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:49.874651  1792 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:49.875133  1785 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:49.875511 28902 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52676 (request call id 245) took 26656 ms (client timeout 15000 ms). Trace:
I20260709 13:12:49.875552  1787 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:49.876340  1787 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:49.876718  1906 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:49.877966  1906 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.879014 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.879442  1908 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15.
I20260709 13:12:49.880319  1785 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:49.881320 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:49.872504  1786 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:49.884495  1786 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:49.889106  1693 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:49.889356 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 530518us
I20260709 13:12:49.889961  1693 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:49.882293 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 811) took 1314 ms. Trace:
I20260709 13:12:49.892356 29158 rpcz_store.cc:276] 0709 13:12:48.567212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.567433 (+   221us) service_pool.cc:224] Handling call
0709 13:12:48.573009 (+  5576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:48.705543 (+132534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:49.881566 (+1176023us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54004010d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:49.881945 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303229696}
I20260709 13:12:49.902429  1785 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.904373  1785 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:49.905750  1623 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:49.895807 29083 pending_rounds.cc:107] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:12:49.893481  1786 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:49.847148  1950 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:49.909773 29083 raft_consensus.cc:2889] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:49.895426  1567 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.918004  1821 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:49.930588  1792 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:49.910713  1785 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:49.935993 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 106534us
I20260709 13:12:49.942348 29713 raft_consensus.cc:1240] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:49.945538 29715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49414 (request call id 746) took 2099 ms (client timeout 1817 ms). Trace:
W20260709 13:12:49.946734 29715 rpcz_store.cc:269] 0709 13:12:47.845948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.849474 (+  3526us) service_pool.cc:224] Handling call
0709 13:12:49.944985 (+2095511us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b5400650ed4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:49.945229 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1493737728}
W20260709 13:12:49.882416 28902 rpcz_store.cc:269] 0709 13:12:23.218547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.218692 (+   145us) service_pool.cc:224] Handling call
0709 13:12:49.875273 (+26656581us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:38.674126 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:38.714065 (+ 39939us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:38.714118 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:12:38.714142 (+    24us) tablet.cc:662] Decoding operations
0709 13:12:38.715681 (+  1539us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:38.715792 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:38.715824 (+    32us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:12:38.822162 (+106338us) tablet.cc:701] Row locks acquired
0709 13:12:38.822210 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:12:38.822377 (+   167us) write_op.cc:270] Start()
0709 13:12:38.822528 (+   151us) write_op.cc:276] Timestamp: P: 1783602758822314 usec, L: 0
0709 13:12:38.822552 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:12:38.823212 (+   660us) log.cc:844] Serialized 6255 byte log entry
0709 13:12:39.079214 (+256002us) op_driver.cc:464] REPLICATION: finished
0709 13:12:39.080429 (+  1215us) write_op.cc:301] APPLY: starting
0709 13:12:39.080508 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:39.346365 (+265857us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:39.346544 (+   179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:40.065496 (+718952us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:40.065966 (+   470us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:12:40.066047 (+    81us) write_op.cc:312] APPLY: finished
0709 13:12:40.066750 (+   703us) log.cc:844] Serialized 148 byte log entry
0709 13:12:40.068480 (+  1730us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:40.068786 (+   306us) write_op.cc:454] Released schema lock
0709 13:12:40.068855 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:40.069543 (+   688us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:40.069588 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6455196,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":974,"apply.run_cpu_time_us":29634,"apply.run_wall_time_us":988939,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71007,"num_ops":16,"prepare.queue_time_us":160929,"prepare.run_cpu_time_us":32564,"prepare.run_wall_time_us":275147,"raft.queue_time_us":2945,"raft.run_cpu_time_us":1233,"raft.run_wall_time_us":1233,"replication_time_us":256507,"thread_start_us":226641,"threads_started":4,"wal-append.queue_time_us":165577,"wal-append.run_cpu_time_us":6192,"wal-append.run_wall_time_us":2071675}]]}
W20260709 13:12:49.955487  1920 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.060s	sys 0.104s
I20260709 13:12:49.913048  1693 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:49.975970  1920 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.061s	sys 0.105s
I20260709 13:12:49.984899 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:49.981184 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:49.979663  1949 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:49.975565  1567 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.015762 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35272 (request call id 2033) took 2094 ms. Trace:
I20260709 13:12:50.011497 29084 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 15. Ops: []
W20260709 13:12:50.008256 29001 consensus_meta.cc:287] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.643s	user 0.003s	sys 0.019s
I20260709 13:12:50.023729 29001 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:12:50.039217 29082 raft_consensus.cc:1275] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:12:50.039542 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.016353 29159 rpcz_store.cc:276] 0709 13:12:47.917018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.919034 (+  2016us) service_pool.cc:224] Handling call
0709 13:12:47.919366 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:48.107956 (+188590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.004157 (+1896201us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54003f6bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:50.011157 (+  7000us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393135488}
I20260709 13:12:50.045521  1950 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.045979  1567 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:50.046773  1948 log.cc:927] Time spent T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:12:50.051011 29079 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:12:50.056275  1787 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.087072  1909 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:50.049170  1949 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:50.098613 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 122271us
I20260709 13:12:50.100533  1951 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:50.128937  1949 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.137552  1567 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:50.141181  1567 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.143258 28976 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:50.148895  1912 consensus_queue.cc:1048] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
W20260709 13:12:50.154753 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 273600us
I20260709 13:12:50.159621  1787 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:50.159653  1897 consensus_meta.cc:287] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.947s	user 0.059s	sys 0.115s
I20260709 13:12:50.156563  1693 raft_consensus.cc:993] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:12:50.169512  1567 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.249473  1567 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.220427 29010 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
W20260709 13:12:50.220841  1960 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.055s	user 0.000s	sys 0.019s
I20260709 13:12:50.249985  1693 raft_consensus.cc:1081] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:12:50.172318  1836 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.170626  1787 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:50.222708 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.252087  1836 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.255795  1836 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.256560 29083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2400) took 1151 ms. Trace:
I20260709 13:12:50.257006 29083 rpcz_store.cc:276] 0709 13:12:49.105159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.227646 (+122487us) service_pool.cc:224] Handling call
0709 13:12:49.227906 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.914643 (+686737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.256229 (+341586us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54000465d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:50.256501 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1417036160}
W20260709 13:12:50.223450  1745 consensus_meta.cc:287] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.033s	user 0.017s	sys 0.018s
I20260709 13:12:50.223181 29708 consensus_queue.cc:237] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.226663 29050 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.152873  1949 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 13:12:50.232546 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 115223us
I20260709 13:12:50.233837  1123 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:50.234427  1954 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.086s	user 0.002s	sys 0.001s
I20260709 13:12:50.172255  1917 consensus_queue.cc:1059] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:50.250047 29084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 1821) took 1097 ms. Trace:
I20260709 13:12:50.268594 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:50.265415 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.270905 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.272011  1567 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:50.272523  1567 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:50.268844 29084 rpcz_store.cc:276] 0709 13:12:49.152473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.841983 (+689510us) service_pool.cc:224] Handling call
0709 13:12:49.842639 (+   656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:50.020772 (+178133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.249788 (+229016us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54000460d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:50.249971 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367898496}
I20260709 13:12:50.271811  1745 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.260639 29079 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:12:50.289188 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 166297us
I20260709 13:12:50.294773  1567 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.296833  1567 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:50.299656  1745 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:50.275976  1693 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:50.328357 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 53481.8us
W20260709 13:12:50.329809 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.333619 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 50707.5us
W20260709 13:12:50.339491 28973 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.339632  1839 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:50.340557  1839 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.348371  1716 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.349032  1716 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.350874  1795 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:12:50.348577 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.351773  1716 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.356230  1693 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.361811  1693 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.352519 28976 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:50.363209  1949 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:50.351938  1745 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:50.358515  1909 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.332510  1123 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.376226  1909 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:50.379051  1123 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:50.343602  1839 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.336493  1917 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:50.373584  1949 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:12:50.374240 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 115186us
W20260709 13:12:50.387671 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 144719us
I20260709 13:12:50.388334 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.390045 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.394973  1964 consensus_queue.cc:1048] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:50.396294  1795 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:50.397183  1795 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:12:50.385859 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.366744 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 204506us
I20260709 13:12:50.398801  1740 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:50.385368  1950 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:12:50.399701 29682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.411584 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.413311  1906 raft_consensus.cc:3055] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:50.413487 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.411926 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.413760  1906 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:50.416275  1906 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.417675  1908 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:50.418944  1839 consensus_queue.cc:1048] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:50.424387 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:50.425329 29680 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:50.398074  1859 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.429678 29007 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:50.429533 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:50.429971  1827 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:50.434803  1795 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:50.435360  1827 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:50.431486  1693 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:12:50.436484  1827 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.437229 29003 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:12:50.430167 29004 raft_consensus.cc:2410] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:12:50.430061  1917 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:50.414475  1950 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:50.432595 29007 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:50.440338 29080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2402) took 1225 ms. Trace:
I20260709 13:12:50.440447  1151 raft_consensus.cc:993] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:12:50.440951 29007 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.440754 29080 rpcz_store.cc:276] 0709 13:12:49.195101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.844162 (+649061us) service_pool.cc:224] Handling call
0709 13:12:49.844765 (+   603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:49.845483 (+   718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:49.845524 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:49.846525 (+  1001us) log.cc:844] Serialized 26 byte log entry
0709 13:12:50.085032 (+238507us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:50.085088 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.085196 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:50.394539 (+309343us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b24000fea38. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f41125da9c4 00007f41125d8acb 00007f4112793dc9 00007f4112793a55 00007f4112792fb0 00007f4115409488 00007f4115409be8 00007f41153e1398 00007f41153dc76a 00007f41153dbbb6
0709 13:12:50.414052 (+ 19513us) raft_consensus.cc:1685] finished
0709 13:12:50.414075 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:50.420806 (+  6731us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33410,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":305,"spinlock_wait_cycles":67385984,"thread_start_us":237923,"threads_started":1,"wal-append.queue_time_us":290156}
I20260709 13:12:50.442224 29007 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:50.442685 29010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34922 (request call id 2704) took 1524 ms (client timeout 1918 ms). Trace:
W20260709 13:12:50.443076 29010 rpcz_store.cc:269] 0709 13:12:48.918330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.918490 (+   160us) service_pool.cc:224] Handling call
0709 13:12:50.442476 (+1523986us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540012f554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:50.442596 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2858319232}
W20260709 13:12:50.445540 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 74584.9us
I20260709 13:12:50.451529 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.452368  1917 raft_consensus.cc:2764] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 13:12:50.453574  1947 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.475s	user 0.004s	sys 0.000s
I20260709 13:12:50.461763 29711 raft_consensus.cc:1240] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
W20260709 13:12:50.461889 29048 connection.cc:256] RPC call timeout handler was delayed by -1.0532872320001161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:12:50.462399 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.464251 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 50826.6us
I20260709 13:12:50.465258 29711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 756) took 1934 ms. Trace:
I20260709 13:12:50.465878 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.465698 29711 rpcz_store.cc:276] 0709 13:12:48.530683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.530798 (+   115us) service_pool.cc:224] Handling call
0709 13:12:48.531051 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:50.462336 (+1931285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.464985 (+  2649us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b5400014854. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:50.465188 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4439265536}
I20260709 13:12:50.467275  1771 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:50.468523 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.468989 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.469447  1771 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.469993  1771 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.439872  1950 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.472234  1906 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:12:50.472854  1859 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.473651 29680 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:50.474561  1771 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.475198  1970 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:50.475689  1970 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.475950  1859 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.476202  1970 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:50.478556 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:50.472327  1950 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:50.480362 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:50.480448  1693 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:50.482033  1972 raft_consensus.cc:1081] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
W20260709 13:12:50.482649 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54332.2us
I20260709 13:12:50.483625  1567 consensus_queue.cc:1059] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:50.484695  1787 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:50.485147  1787 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:50.486366  1787 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:50.481992 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 59966.7us
I20260709 13:12:50.508059 29010 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:50.508162 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:50.509783 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:50.511077 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:50.511777  1821 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:50.512557  1821 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:50.513218  1821 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:50.514707  1851 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.515342  1851 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:50.519081 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.518271 29683 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:50.467876 29050 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:50.519852 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 52162.1us
I20260709 13:12:50.520609 29050 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:50.523181  1787 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:50.524214  1795 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:50.527498  1803 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:50.532308  1970 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.534703  1970 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:50.536195  1971 log.cc:927] Time spent T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:12:50.428618  1745 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.539789  1567 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.540463  1567 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.541571  1745 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.553313 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:50.558019 29010 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:50.543603  1771 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.577009 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:50.561177  1851 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.561694  1787 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.552845  1151 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:12:50.595942 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:50.573047  1567 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.568316  1795 raft_consensus.cc:2764] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:12:50.608266  1827 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.608846  1827 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.611287  1949 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.611605  1827 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.612208 29077 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:50.615099 29005 raft_consensus.cc:1275] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:50.615558 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:50.616396 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:50.617128 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:50.617383 29008 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:12:50.617583 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:50.617162 29711 raft_consensus.cc:2393] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 13: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:50.618331 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:50.618635  1821 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:50.566015  1897 consensus_queue.cc:1059] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:12:50.619048 29002 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.619251 29001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a08e186ea0749ae979329103591a933"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:12:50.626106 29084 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:12:50.619048  1821 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.629364  1821 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.631572 29006 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:50.619968 29001 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:50.635557 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:50.611898  1949 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.640630 29001 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:50.646942 29002 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:12:50.648423 29010 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 15.
I20260709 13:12:50.652621 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:50.654115 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:50.655432 29007 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:12:50.644438 29008 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:50.656416  1492 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:12:50.657598  1716 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.650774 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:50.658581  1966 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:50.661919  1963 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.666064  1949 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.681851 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:50.668140 29003 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:50.668510  1906 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.633602  1771 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.653116 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:50.671859  1909 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.676551 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:50.680794 29007 pending_rounds.cc:85] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 10
I20260709 13:12:50.694092 29007 pending_rounds.cc:107] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:12:50.690815 29001 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.694461 29007 raft_consensus.cc:2889] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:50.684752 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:50.685456  1982 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:50.688256 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 56030.9us
I20260709 13:12:50.688388 29680 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:50.690987 29708 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:12:50.692651 29078 raft_consensus.cc:1275] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:50.694159 29004 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:12:50.698778 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:50.704627  1969 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.653477  1787 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.705258  1969 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.706621  1827 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.707140  1827 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.686094  1716 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.708472  1787 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.708554  1968 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.709164  1968 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.709908  1716 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.711185  1968 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:50.708164  1836 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:50.711702  1836 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.712118  1836 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:50.714483  1982 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.715847  1906 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:50.693410  1966 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:50.717856 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.688834  1963 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.702975  1859 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:50.720183  1859 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 15 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:50.721103  1859 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.730199  1740 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:50.730765  1740 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.734769 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.735400 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:50.736402 29154 raft_consensus.cc:1240] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: [11.10-12.11]
I20260709 13:12:50.697409  1912 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:50.737638 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.737491  1912 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:50.738281 29048 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.918s (SENT)
I20260709 13:12:50.738989 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.740368  1771 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.741150 29151 raft_consensus.cc:1240] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:50.742892 29150 raft_consensus.cc:1240] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:12:50.735451 29152 raft_consensus.cc:1813] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Handling vote request from an unknown peer 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:50.745090 29001 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.745652 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:50.743724 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:50.719828  1785 raft_consensus.cc:3055] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:50.746426 29708 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:50.702214  1949 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:50.761521  1785 raft_consensus.cc:740] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:50.770771 29076 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
I20260709 13:12:50.767652 29152 raft_consensus.cc:2393] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 8: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:50.768836 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 119945us
I20260709 13:12:50.769699 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.735445  1961 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:50.743826  1970 consensus_queue.cc:1048] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:12:50.788610 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:50.783923 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 59575.8us
W20260709 13:12:50.785271 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 54027.2us
I20260709 13:12:50.792711 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:50.805265 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.765843  1821 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.807507 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:50.806779 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:50.815317 29155 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:50.840739 29007 raft_consensus.cc:2393] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 12: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:50.842172 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.846585 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.808259  1827 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.801582  1839 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:12:50.838778  1787 raft_consensus.cc:2764] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:12:50.866472  1123 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:50.849808  1771 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:50.836549  1851 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:12:50.791826  1961 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:50.871068 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.882674 29008 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31c5d53651b5426780911903843f0bc3"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:12:50.889274 29004 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:50.889353 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.890640  1961 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:50.892689 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 103320us
I20260709 13:12:50.868103  1771 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.894979 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:50.901309  1993 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:12:50.905140 29008 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:50.905498 29006 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:12:50.913811  1821 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.914579 28976 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:50.915812  1977 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:12:50.922238  1966 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.873294  1827 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:50.926604  1909 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.979925 29008 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:50.929618 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:50.930392 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.930681  1982 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.931221 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.931375 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.931728 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:51.002595 29157 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:50.930754 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.934172  1740 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:50.931419 29150 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:50.937050  1983 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:50.937134  1745 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:50.938776  1949 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.941332  1912 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:50.928810 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:50.995532 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:50.896878  1771 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:50.947136 29006 pending_rounds.cc:85] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 6
I20260709 13:12:51.002930 29157 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:51.041996 29006 pending_rounds.cc:107] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 14.7
W20260709 13:12:51.038194 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 239201us
W20260709 13:12:51.000100 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.042342 29006 raft_consensus.cc:2889] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:51.049449 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 134944us
I20260709 13:12:51.044521  1740 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:12:50.979091  1827 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:51.057368 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.057870 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.067792  1906 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:51.047843  1123 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:12:51.072212  1949 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.074874  1949 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:51.075363  1988 log.cc:927] Time spent T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 13:12:51.126387 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.128423 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 66754.5us
W20260709 13:12:51.139555 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.129262 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:51.146956 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:50.860072  1785 consensus_queue.cc:260] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.150652 29152 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:51.151850 29007 raft_consensus.cc:1275] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:12:51.162545  2013 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.163046 29077 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:12:51.163655  1740 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:51.164450  1969 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:51.167143  1963 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:51.216392 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.061734  1827 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:51.184937 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.185596 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.221330 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:51.222530  1151 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:51.223097  1151 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:51.223165  2016 consensus_queue.cc:1048] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:12:51.224665  1961 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:51.226583 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:51.227244 29009 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 10.
I20260709 13:12:51.228431 29009 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
W20260709 13:12:51.231395  1996 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:12:51.222545 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:51.205442 29007 pending_rounds.cc:85] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 8
I20260709 13:12:51.235582 29007 pending_rounds.cc:107] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:12:51.235920 29007 raft_consensus.cc:2889] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:51.185496  1836 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.180375 29157 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.238313 29157 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:51.158522  1859 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.156926  1787 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:51.223112  2013 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.273310  1859 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.275238  2013 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:51.278035 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 109061us
I20260709 13:12:51.205049  1949 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.288450 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.289074 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:51.298624 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 150610us
I20260709 13:12:51.300060 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:51.292078 29081 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:51.304759 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.278899 29122 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:51.306747 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.312917 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:51.316838 29010 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.082s	sys 0.102s
I20260709 13:12:51.316712 29122 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:51.316737 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.317685 29010 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.082s	sys 0.103s
I20260709 13:12:51.319748 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.320180 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:51.321982 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 134032us
I20260709 13:12:51.345431 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.303705  1982 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:51.231379  1827 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.312956  1151 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.354708  1740 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:51.356812  1821 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:51.357434 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.358287 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:51.358934 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.359436 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:51.344921  2016 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:51.345145  1970 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:12:51.364416 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.368883 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:51.347970  1961 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:51.378108  1771 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.383548  1745 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:51.346354 29079 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:51.351213 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.395174 29156 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:51.380023  1785 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:51.399142 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 133164us
I20260709 13:12:51.416662 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:51.447924 29085 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 16: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:12:51.419478 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.422188 29004 raft_consensus.cc:2410] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:12:51.371845 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:51.423509 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.428375 29006 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 16: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:51.347322  1969 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:51.459026 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.468582 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.336125  1949 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.429045  1961 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.476449 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:51.483609 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:51.428778  1966 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.303135  1859 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:51.510041 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 143043us
I20260709 13:12:51.484342 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.512645 29085 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:51.482740 29048 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:51.513880 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.514112 29076 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:51.471944  1151 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:51.520184 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.520486 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.526271 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.521927 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:51.457345  1821 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.475946  1969 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:51.483451  1961 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:51.535108 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 232744us
I20260709 13:12:51.537619 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:51.540684 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58049.7us
I20260709 13:12:51.541950 29003 raft_consensus.cc:2393] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 16: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:51.545531  1909 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.554590 29003 raft_consensus.cc:1240] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 12. Current term is 14. Ops: []
I20260709 13:12:51.510042  2013 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:12:51.554734  1909 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.556510  1982 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:12:51.557015  1909 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:51.557406  1982 consensus_queue.cc:1048] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:51.532374  1950 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:51.559621  1950 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.560037  1950 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:51.560329  1836 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:51.561388  2005 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
W20260709 13:12:51.562467  2003 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:12:51.564534  1969 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.566861 29005 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:51.555997 29003 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:51.568954  1771 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:51.514405  1966 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.569552  1745 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:51.569878 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.569957  1745 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:51.533579  1795 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:51.573238 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:51.597112 29083 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:51.547554 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.570729 29156 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 18.
W20260709 13:12:51.597780 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.598503 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.599551 29085 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:12:51.598986 29008 consensus_meta.cc:287] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.002s
I20260709 13:12:51.518613  1972 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:12:51.644069 29079 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 18.
I20260709 13:12:51.603547 29008 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:51.646417 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.648540 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:51.564564  1949 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:51.554840  1827 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:51.647228 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61820.2us
I20260709 13:12:51.658427 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.660599 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:51.667059 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.667207  1949 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:51.667675  1949 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.668073  1949 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:51.676206 29008 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:51.684553  2012 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.685148  2012 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.685873 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:51.686480 29010 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:51.687397  2012 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:51.688658 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:51.689581 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:51.690155 29708 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:51.689586 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:51.694433  1966 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:51.694794 29010 raft_consensus.cc:2468] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
I20260709 13:12:51.695904 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:51.700337 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:51.700682 29151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:51.701342 29151 raft_consensus.cc:606] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:51.707311 29157 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:51.726429 29083 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
W20260709 13:12:51.733511 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.649197  1771 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.748209 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:51.748512 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:51.749092 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.751107 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:51.751864 29083 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:12:51.750036 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 211423us
W20260709 13:12:51.725564  1851 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.012s	sys 0.026s
W20260709 13:12:51.755915  1851 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.013s	sys 0.027s
I20260709 13:12:51.760383 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.761387 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:51.764671 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:51.748869 29713 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:51.722124  1917 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.774639 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.785506 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:51.767803 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:51.749768 29084 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:51.784772 29683 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:51.785912 29083 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:12:51.766510  1906 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.798992 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:51.793481 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.808730 29157 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:51.809113 29157 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:51.799615 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:51.804410  1917 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.804701 29680 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:12:51.805126 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.741456  1897 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:12:51.766283  2013 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.809839 29157 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.776481  1787 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.811265 29708 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:51.829186 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:51.840219 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:51.844473 29156 raft_consensus.cc:3060] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:12:51.845553 29157 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:12:51.845639 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:12:51.818480 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 278205us
I20260709 13:12:51.845633 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:51.760640  1771 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.799309  1859 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:51.858387 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:51.809161  2021 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:51.811964  1993 raft_consensus.cc:2804] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:51.748872  1745 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:51.905102 29152 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:51.905599 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:51.895671 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 46519.6us
I20260709 13:12:51.861431  1827 raft_consensus.cc:2804] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:51.813416  1906 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.915683 29717 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:12:51.857431  1859 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.921200 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.845412  1787 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.888396  2021 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:51.927961 29157 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:12:51.931569 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:51.956842 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.958199 29048 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:51.961915 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 276592us
I20260709 13:12:51.914650  1795 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:51.905565  1993 raft_consensus.cc:697] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:51.988009 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.854432  2013 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:51.999270 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:51.999421 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:52.000368 29157 raft_consensus.cc:2368] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 17. Current term is 18.
I20260709 13:12:51.933434  1859 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:51.954486  1827 raft_consensus.cc:697] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:51.916199  1821 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:52.032344 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.960644  2021 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:52.024930 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:51.916965  1950 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.072201  1897 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:52.072815  1897 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.073300  1897 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:52.073938  1950 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.079272 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:52.081161 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:52.082626 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.066780  1859 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:52.092917  1771 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.093618  1859 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:51.997793  2012 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.096290  2012 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.098743  2012 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.099341  2012 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:52.100481  1993 consensus_queue.cc:237] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:52.103981  1897 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.106624  1897 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:52.108754  2016 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.000s
I20260709 13:12:52.109169  2016 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:52.110464 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:52.114794 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 160528us
I20260709 13:12:51.987044  1787 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.037038 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:51.919135  1912 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Committing config change with OpId 8.6: config changed from index 4 to 6, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:52.042596 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:52.043355 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:52.123965 29001 raft_consensus.cc:2368] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 17. Current term is 18.
I20260709 13:12:52.048605  1827 consensus_queue.cc:237] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:52.049302  1906 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.049958  1917 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:52.031769  1151 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.507s	user 0.003s	sys 0.006s
I20260709 13:12:52.056636 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:52.059758 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:52.062412 29713 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:12:52.130317 29715 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:52.062961 29076 consensus_meta.cc:287] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Time spent flushing consensus metadata: real 0.530s	user 0.005s	sys 0.001s
I20260709 13:12:52.131184 29076 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:52.069391  2013 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.133205 29150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94045d5a8709418e82919c9fbf804582"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
W20260709 13:12:52.071645  2024 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 13:12:52.072072 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 74823.2us
I20260709 13:12:52.079910 29151 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
W20260709 13:12:52.036556  2023 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.381s	user 0.001s	sys 0.000s
I20260709 13:12:52.095324  1771 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:12:52.122977 29085 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:52.132738  1795 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:52.166884 29006 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:52.167922 29001 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
W20260709 13:12:52.126406 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 322621us
W20260709 13:12:52.126910 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 321191us
I20260709 13:12:52.127007 29156 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:12:52.131486 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:52.144742 28975 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:52.145258 28976 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:52.175503 29708 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 19: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:52.176249 29150 raft_consensus.cc:606] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:52.176627 29150 raft_consensus.cc:614] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:12:52.178786 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 364633us
W20260709 13:12:52.179710 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 375032us
I20260709 13:12:52.156692  1897 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.156430  1787 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.174108 29682 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:52.184190 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:52.188642 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:52.189368 29076 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:52.194646 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:52.197284 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 368147us
I20260709 13:12:52.199604 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:52.201910 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:52.159965  1950 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 13:12:52.247514 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 67944us
I20260709 13:12:52.248199 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:52.248930 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:52.250339 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 69814.6us
I20260709 13:12:52.269781 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 953) took 1100 ms. Trace:
I20260709 13:12:52.206542  2034 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:12:52.245062  1785 consensus_meta.cc:287] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.776s	user 0.001s	sys 0.008s
I20260709 13:12:52.291092 29002 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
I20260709 13:12:52.186714  1897 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.296357 29001 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:12:52.296783 29001 pending_rounds.cc:85] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 6
I20260709 13:12:52.297065 29001 pending_rounds.cc:107] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:12:52.297384 29001 raft_consensus.cc:2889] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:52.273775  2023 log.cc:927] Time spent T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:12:52.202879  1771 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:52.274124  1949 consensus_meta.cc:287] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.005s
I20260709 13:12:52.274803 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:52.191744  1787 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.270336 29009 rpcz_store.cc:276] 0709 13:12:51.168917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.229234 (+ 60317us) service_pool.cc:224] Handling call
0709 13:12:51.229523 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.229886 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.229913 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.230333 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:12:51.612207 (+381874us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:51.612264 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.612360 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.269517 (+657157us) raft_consensus.cc:1685] finished
0709 13:12:52.269546 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.269708 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8015,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":28458,"thread_start_us":381440,"threads_started":1,"wal-append.queue_time_us":544508}
I20260709 13:12:52.302942 29006 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:52.308166 29158 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:12:52.308871 29051 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:52.334007 29156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8340da85294490fb88f62f80393531f"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:52.334805 29156 raft_consensus.cc:606] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:52.277266  2034 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:52.275462  1151 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:12:52.289340  2037 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:12:52.343964 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 273055us
I20260709 13:12:52.318615  1785 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:52.358599  1917 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.358703  1906 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.359166  1917 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.359243  1906 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.335760  1909 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:12:52.364889 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 180747us
I20260709 13:12:52.375273 28976 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:52.297801  2006 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:12:52.337487  2041 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:52.379969 29076 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 19.
I20260709 13:12:52.389034 29125 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:52.322893  2021 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:52.343153  2044 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:52.331555  1949 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.321285  1771 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.376850  1785 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:52.392707  1909 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:52.403056  1950 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.429175 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:52.397657  2010 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
W20260709 13:12:52.441066 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 235439us
I20260709 13:12:52.441290 29048 leader_election.cc:400] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 17. Current term is 18.
I20260709 13:12:52.441603 29048 leader_election.cc:403] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:52.443181 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 298535us
I20260709 13:12:52.444300 28975 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:52.452845 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 258253us
I20260709 13:12:52.420497  2047 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.457775 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33750 (request call id 950) took 1609 ms. Trace:
I20260709 13:12:52.461290 29002 rpcz_store.cc:276] 0709 13:12:50.848860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.873999 (+ 25139us) service_pool.cc:224] Handling call
0709 13:12:52.457614 (+1583615us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54001caf54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:52.457721 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3208665984}
I20260709 13:12:52.464354 29002 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:12:52.466779 29002 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:12:52.468122 29002 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:52.411222  1123 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:12:52.473701 29711 raft_consensus.cc:1240] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:52.475402 29713 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:52.487408 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:52.513151  2052 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.513715  2048 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:52.514145  1968 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:12:52.529652 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 75905.3us
I20260709 13:12:52.531807 29009 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:52.537176 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 49952.6us
W20260709 13:12:52.533097 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45365.5us
I20260709 13:12:52.538460 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:52.541500 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:52.542191 29081 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:52.516197  2013 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.546269 29153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 826) took 1155 ms. Trace:
I20260709 13:12:52.558743 29002 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:52.439821  1950 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:52.597365 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 128278us
W20260709 13:12:52.597363 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 153272us
I20260709 13:12:52.456501  1982 consensus_queue.cc:1059] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:12:52.599646 28975 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:52.596529 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:52.604645 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 163473us
I20260709 13:12:52.610118 29078 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:52.556425 29153 rpcz_store.cc:276] 0709 13:12:51.198059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.198255 (+   196us) service_pool.cc:224] Handling call
0709 13:12:51.198555 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.232152 (+ 33597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.232184 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.232651 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:12:51.331927 (+ 99276us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:51.355574 (+ 23647us) log.cc:844] Serialized 14 byte log entry
0709 13:12:51.370404 (+ 14830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.370487 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.353075 (+982588us) raft_consensus.cc:1685] finished
0709 13:12:52.353095 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.353211 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27532,"prepare.queue_time_us":23846,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":50169,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":28117,"thread_start_us":98813,"threads_started":1,"wal-append.queue_time_us":296170}
I20260709 13:12:52.430295  2034 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.611397 29002 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Committing config change with OpId 8.6: config changed from index 4 to 6, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:52.517794  2047 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.592315  1964 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:12:52.627118  2065 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:52.631299  1821 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:52.551925  2041 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:12:52.593935  1785 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.627431  1897 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.631585  2045 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.592968  2013 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.636299  1787 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:52.679674  2036 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.339s	user 0.001s	sys 0.001s
W20260709 13:12:52.681267  2033 log.cc:927] Time spent T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.378s	user 0.001s	sys 0.000s
I20260709 13:12:52.555555  2052 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.682564  1950 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:52.686620  2060 log.cc:927] Time spent T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:12:52.678705  2034 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.683197  2053 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:12:52.683961  2049 log.cc:927] Time spent T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:12:52.683113  2008 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:52.684329  1906 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.681174  1917 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.705474 29078 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:52.616067  1909 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:52.706284 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 105949us
I20260709 13:12:52.699148  2068 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 17. Current term is 18.
W20260709 13:12:52.706908  2040 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:12:52.736505  2047 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.706725  2002 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:52.737149  2002 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:52.737397 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34922 (request call id 2747) took 1008 ms. Trace:
I20260709 13:12:52.737479  2002 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:52.737851 29010 rpcz_store.cc:276] 0709 13:12:51.728591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.728705 (+   114us) service_pool.cc:224] Handling call
0709 13:12:51.731479 (+  2774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.731822 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.731858 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.761182 (+ 29324us) log.cc:844] Serialized 26 byte log entry
0709 13:12:52.134633 (+373451us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:52.405095 (+270462us) log.cc:844] Serialized 14 byte log entry
0709 13:12:52.412190 (+  7095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:52.412259 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.737189 (+324930us) raft_consensus.cc:1685] finished
0709 13:12:52.737211 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.737332 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7515,"prepare.queue_time_us":307174,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":149206,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":40488,"thread_start_us":570025,"threads_started":2,"wal-append.queue_time_us":490903}
W20260709 13:12:52.739161  1961 consensus_meta.cc:287] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 1.143s	user 0.011s	sys 0.000s
I20260709 13:12:52.728771  1906 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.716894  1949 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.739977  1906 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.739632  1961 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.742969 29155 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:52.745216  1961 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:52.746227  2075 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.747535  2075 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.755604  2075 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.755992  1966 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:52.756487  1966 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:52.757333  1966 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:52.758790 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 149200us
I20260709 13:12:52.712949  2021 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.764801 28976 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:52.755950  1906 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.703493  2045 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.772325 29082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39900 (request call id 1000) took 1225 ms. Trace:
I20260709 13:12:52.774441 29082 rpcz_store.cc:276] 0709 13:12:51.546928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.548728 (+  1800us) service_pool.cc:224] Handling call
0709 13:12:51.571048 (+ 22320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.584871 (+ 13823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.584903 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.651203 (+ 66300us) log.cc:844] Serialized 494 byte log entry
0709 13:12:51.965540 (+314337us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:51.965592 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.965682 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.772111 (+806429us) raft_consensus.cc:1685] finished
0709 13:12:52.772131 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.772270 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":281139,"threads_started":1,"wal-append.queue_time_us":493059}
I20260709 13:12:52.761819  1966 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:52.781667 29158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2737) took 1209 ms. Trace:
I20260709 13:12:52.782055 29158 rpcz_store.cc:276] 0709 13:12:51.569487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.595865 (+ 26378us) service_pool.cc:224] Handling call
0709 13:12:52.778731 (+1182866us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54003f7ad4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:52.778874 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137922304}
I20260709 13:12:52.784174  2053 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.784660  2053 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:52.775820  1983 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:12:52.770836  2016 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:12:52.793546  2010 raft_consensus.cc:3055] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:52.793932 29081 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:52.794301  2010 raft_consensus.cc:740] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:52.878937 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61675.4us
I20260709 13:12:52.881474 29683 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:52.881959 29083 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:52.896746 29711 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:12:52.901366  2079 log.cc:927] Time spent T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:12:52.901790  2052 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:52.901964 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 103527us
I20260709 13:12:52.902700 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:52.902606  1983 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:52.904786  2046 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:52.905452  2045 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.896541  2076 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:12:52.907935  2076 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:52.908310  2076 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:52.909152  2076 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.909160  1492 raft_consensus.cc:2804] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:52.910512  2076 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:52.901288  1123 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:52.890365  1917 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.893882  1795 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:52.950001 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:52.950891 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 175592us
I20260709 13:12:52.913388  1123 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:52.943976  1492 raft_consensus.cc:697] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 19 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:52.795815  2010 consensus_queue.cc:260] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:52.964586 29078 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:52.968228 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:52.968621 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:52.948274  1827 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:12:52.950693  1917 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:52.976492 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:52.979749  1492 consensus_queue.cc:237] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:52.952400  1795 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:52.994499 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.973376  2013 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:52.977133  2010 raft_consensus.cc:3060] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:53.011471 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:53.000080  1912 consensus_meta.cc:287] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.003s
W20260709 13:12:53.018227 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46441.6us
I20260709 13:12:53.011160  1917 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:53.020520 29084 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:53.021066  1795 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:53.021668 29084 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 16. Current term is 18. Ops: []
I20260709 13:12:53.027675 29004 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:12:53.029788 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:53.030568 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.033228 29084 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:53.031075 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:53.035362 29005 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:52.978116  1123 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.056174  1912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:51794 (request call id 12) took 2901 ms. Trace:
I20260709 13:12:53.056794  1912 rpcz_store.cc:276] 0709 13:12:50.161991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.164054 (+  2063us) service_pool.cc:224] Handling call
0709 13:12:50.426704 (+262650us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b2c0004cb58. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f4114149a0a 00007f4114149693 00007f4114149631 00007f411587c3f1 00007f41157b59fe 00007f411583c749 00007f411583d76a 00007f411583c91b 00007f411584803c 00007f41158476b3
0709 13:12:50.448390 (+ 21686us) log.cc:844] Serialized 494 byte log entry
0709 13:12:50.598261 (+149871us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b18004cedc0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f411579ab0b 00007f411581fb63 00007f4115847752 00007f411583f9fa
0709 13:12:50.614094 (+ 15833us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b54007317a0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157c1b26 00007f41157bb49a 00007f41157ba714 00007f411579aeaa 00007f41157a1d53 00007f41157a1cd9 00007f41157a1c69 00007f41157a1c31 00007f41157a0dfd 000000000038f354
0709 13:12:50.685000 (+ 70906us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b4c0027d700. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f4113b0da20 00007f4113b1ea29 00007f4113b1f13f 00007f4113ae32ab 00007f4113b10f3c 00007f4113b113fe 00007f4113d9e50b 00007f411579d381 00007f411579b26b 00007f41157a1d53 00007f41157a1cd9
0709 13:12:53.056036 (+2371036us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21647,"raft.queue_time_us":45213,"raft.run_cpu_time_us":4384,"raft.run_wall_time_us":116859,"spinlock_wait_cycles":74540800,"thread_start_us":112932,"threads_started":1,"wal-append.queue_time_us":246119,"wal-append.run_cpu_time_us":3596,"wal-append.run_wall_time_us":1487366}
W20260709 13:12:52.972074  2046 consensus_queue.cc:1175] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ca45420e033847489ad989c1c6eed3ef" responder_term: 13 status { last_received { term: 13 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:53.062139 29076 raft_consensus.cc:2955] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Committing config change with OpId 8.6: config changed from index 4 to 6, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:12:53.126685 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:53.127328 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.088006  2083 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:53.037196  2002 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.094485  2053 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.265057 29005 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:53.151700  2083 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.276875  1151 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:53.310774  1151 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.311987  2002 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:53.313657  1151 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:53.318882  1123 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:53.320547 29004 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:53.320802 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:53.321976 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:53.322721 29085 raft_consensus.cc:2393] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 19: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:53.346381 29081 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:12:53.348287 29081 raft_consensus.cc:1240] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:53.355443 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:53.362455 29077 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:12:53.376031  1151 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:53.376997  1151 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.362933 29077 pending_rounds.cc:85] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 7
I20260709 13:12:53.382117 29077 pending_rounds.cc:107] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:12:53.382442 29077 raft_consensus.cc:2889] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:53.398945 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 43626.8us
I20260709 13:12:53.406822  1151 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:53.399873 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.407573  2010 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.408380  2053 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:53.408872  2083 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:53.412592 29050 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:53.416368  1795 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:53.419256 29004 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:53.422942 29050 leader_election.cc:400] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 17. Current term is 18.
I20260709 13:12:53.423593 29050 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:53.417013  2072 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:53.429535 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:53.424889  2013 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:53.424700  2047 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:53.432659 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 96705.1us
W20260709 13:12:53.431712  1966 consensus_meta.cc:287] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.000s
I20260709 13:12:53.419919  2010 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.432107  1906 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:53.435763  1906 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:53.437405  1906 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:53.438073  2010 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:53.434881  2083 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:12:53.440228 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:53.441499  2083 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:53.441713 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.442871 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.444339  1795 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:53.444885  1795 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.446151 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:53.447196  1795 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:53.447790 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:53.448364  2083 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:53.449084  2083 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:53.449115 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:53.449887 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:53.443943 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.451467 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:53.452158 29076 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:53.452314  2010 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:53.453454  2010 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:53.454176  1982 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:53.454645  1982 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:53.455601  1982 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.457726 29125 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.451658  1906 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.459029  2083 raft_consensus.cc:3060] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:53.456701  1982 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:53.461095  2008 consensus_queue.cc:1059] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:12:53.469586  2008 consensus_queue.cc:1243] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer befda4a6c8344c26b0f139e4e1686b3b log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:12:53.468585 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:53.474213  1839 raft_consensus.cc:993] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:12:53.474771  1839 raft_consensus.cc:1081] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
W20260709 13:12:53.478574 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.477661 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:12:53.482033  2038 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 1.178s	user 0.001s	sys 0.017s
I20260709 13:12:53.486423  1740 raft_consensus.cc:3055] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:53.486986  1740 raft_consensus.cc:740] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:53.492825  1740 consensus_queue.cc:260] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.504735  1740 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:53.505925  1745 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:53.507735 29155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2739) took 1937 ms. Trace:
I20260709 13:12:53.508275 29155 rpcz_store.cc:276] 0709 13:12:51.570103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.571826 (+  1723us) service_pool.cc:224] Handling call
0709 13:12:51.604929 (+ 33103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:52.743351 (+1138422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.507438 (+764087us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400055d54. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:53.507659 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2616744320}
I20260709 13:12:53.510484  2008 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:53.511092  2008 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.512760 29006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:12:53.514286 29122 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:53.514539 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:53.530814  2106 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:53.531361  2106 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:53.532366  2106 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.535993  2030 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:53.536960  1966 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:53.537331  1966 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:53.538467  1966 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.544260 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
I20260709 13:12:53.550741  2083 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:53.557328 29708 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:53.557574 29714 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:53.558136  2008 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:53.557628 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:53.558435 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:53.572077 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 110713us
W20260709 13:12:53.582787 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.585806 29711 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:53.586055 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:53.591982  1771 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:53.598963  1740 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:53.601054  1906 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:53.603125 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:53.605208 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:53.606985 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:53.611160 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:53.611691 29155 raft_consensus.cc:2376] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16. Re-sending same reply.
I20260709 13:12:53.622030 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:53.631421 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.635663 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:53.638407 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:53.650429 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:53.626298  2037 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.213s	user 0.026s	sys 0.042s
I20260709 13:12:53.658089 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:53.567915  2083 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:12:53.660555 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:53.662789 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:53.670225 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:53.654441  1693 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:12:53.660758  2037 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.254s	user 0.026s	sys 0.042s
I20260709 13:12:53.655288  1851 raft_consensus.cc:3055] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:53.625466  2047 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:12:53.678344 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:53.679450 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.692353 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:53.673965  1693 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:53.675645  2037 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:53.677135  1851 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:53.701344 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:53.711431 29157 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:12:53.721009  1973 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:53.729414  2102 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.748162  2102 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.747211 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:53.749642 29152 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:53.757652 29152 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:53.758981  2016 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:53.759428  2016 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:12:53.768519  2109 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:12:53.698097  1693 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:53.777469  2037 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:53.780676 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 84802.2us
I20260709 13:12:53.793915 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.797369 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:53.822533 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 838) took 1981 ms. Trace:
I20260709 13:12:53.823029 29157 rpcz_store.cc:276] 0709 13:12:51.841593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.161259 (+319666us) service_pool.cc:224] Handling call
0709 13:12:52.174806 (+ 13547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:53.729242 (+1554436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.822270 (+ 93028us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400419bd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:53.822465 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3532700544}
I20260709 13:12:53.834563 29158 raft_consensus.cc:2468] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 19.
W20260709 13:12:53.840767 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 46987.3us
I20260709 13:12:53.841679 29125 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:53.836066  2101 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:12:53.847585 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:53.848400 29159 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:53.849736 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 119985us
I20260709 13:12:53.853336 29153 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:12:53.838239  2102 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:53.855757 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.856613 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:53.860831 29151 raft_consensus.cc:1275] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:53.861737  2117 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:53.862527 29151 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:12:53.862921 29151 pending_rounds.cc:85] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 11
I20260709 13:12:53.863202 29151 pending_rounds.cc:107] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:12:53.863482 29151 raft_consensus.cc:2889] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:53.876138  2059 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:12:53.879074 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.880254  1745 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:53.843905  1740 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:53.835677  2016 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.885825  2016 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:53.887915 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:53.888113  2030 consensus_queue.cc:1175] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ca45420e033847489ad989c1c6eed3ef" responder_term: 11 status { last_received { term: 10 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:53.888552  2047 consensus_queue.cc:1059] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:53.891408  1771 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:53.891759  2074 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.891985  1771 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:53.892267  2074 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.892951  1771 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.894448  2074 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:53.894933  2074 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.895323  2074 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.896226  2034 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:53.896637  2069 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:53.896656  2034 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
W20260709 13:12:53.900564 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 1.13574e+06us
I20260709 13:12:53.901347 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.902789 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.899600  2034 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:53.903539  2052 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:53.903821  2034 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:53.904281  2052 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:53.904968  2052 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:53.906038 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 963) took 1585 ms. Trace:
I20260709 13:12:53.901363  1966 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:53.906545 29002 rpcz_store.cc:276] 0709 13:12:52.321027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.610863 (+289836us) service_pool.cc:224] Handling call
0709 13:12:52.611119 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:52.611296 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:52.725208 (+113912us) log.cc:844] Serialized 14 byte log entry
0709 13:12:53.905693 (+1180485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:53.905745 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:53.905795 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.905868 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.905977 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":941243,"thread_start_us":238439,"threads_started":1,"wal-append.queue_time_us":299639}
W20260709 13:12:53.908325 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 52653.3us
I20260709 13:12:53.891450  2055 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:53.891443 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2731) took 1744 ms. Trace:
I20260709 13:12:53.914181 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:53.922520 29002 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:53.906064 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:53.930884 29050 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:53.939656 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:53.942689  2019 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:12:53.944543  2056 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:53.944667 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2732) took 1797 ms. Trace:
I20260709 13:12:53.945251  1950 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:53.945099  2056 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.945844  1950 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.946303  2002 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:53.948091  1950 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:53.949553  1851 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:53.950191 29085 raft_consensus.cc:1275] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:12:53.950775  1851 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:53.927336  1964 consensus_queue.cc:1048] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:12:53.932900 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:53.954126 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:53.954679 29005 raft_consensus.cc:3060] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:53.956236 29006 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:53.956626 29006 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:53.961926 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 55934us
W20260709 13:12:53.962564 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 1.36286e+06us
I20260709 13:12:53.962774 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:53.963299 28975 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:53.921083 29003 rpcz_store.cc:276] 0709 13:12:52.146823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.146947 (+   124us) service_pool.cc:224] Handling call
0709 13:12:52.161465 (+ 14518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:52.166620 (+  5155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:52.166648 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:52.186122 (+ 19474us) log.cc:844] Serialized 26 byte log entry
0709 13:12:52.414356 (+228234us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:52.589111 (+174755us) log.cc:844] Serialized 14 byte log entry
0709 13:12:53.890798 (+1301687us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b1800497140. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:12:53.890954 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.891038 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:53.891101 (+    63us) raft_consensus.cc:1685] finished
0709 13:12:53.891208 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.891353 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24219,"prepare.queue_time_us":122600,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":1317659,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":4166,"spinlock_wait_cycles":29056256,"thread_start_us":283426,"threads_started":2,"wal-append.queue_time_us":213772,"wal-append.run_cpu_time_us":4058,"wal-append.run_wall_time_us":1258101}
I20260709 13:12:53.945106 29008 rpcz_store.cc:276] 0709 13:12:52.147170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.147283 (+   113us) service_pool.cc:224] Handling call
0709 13:12:52.161671 (+ 14388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:52.161844 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:52.391125 (+229281us) log.cc:844] Serialized 14 byte log entry
0709 13:12:53.748643 (+1357518us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b1800496fc0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41153e94bc 00007f411273748b 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a
0709 13:12:53.944307 (+195664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:53.944365 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:53.944431 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.944495 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.944610 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":244268,"prepare.queue_time_us":295136,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":197199,"raft.run_cpu_time_us":56,"raft.run_wall_time_us":55,"spinlock_wait_cycles":35182208,"thread_start_us":376004,"threads_started":2,"wal-append.queue_time_us":317379,"wal-append.run_cpu_time_us":1133,"wal-append.run_wall_time_us":1037208}
I20260709 13:12:53.981621 29005 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:12:53.983731 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:53.969805 29048 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.610s (SENT) [suppressed 32 similar messages]
I20260709 13:12:53.925483  2053 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:53.990908 29159 raft_consensus.cc:2468] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:12:53.983029 29680 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:53.993269 28975 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:54.000017 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:54.004454  2065 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:53.973822  2052 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:54.005194  2019 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.005782  2019 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:54.008705  2074 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:54.008901 29157 raft_consensus.cc:1275] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:12:54.009308 29157 pending_rounds.cc:85] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 9
I20260709 13:12:54.009521 29157 pending_rounds.cc:107] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:12:53.973886  1906 raft_consensus.cc:2804] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:54.009725 29157 raft_consensus.cc:2889] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:54.011471 29079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2446) took 1001 ms. Trace:
W20260709 13:12:54.035867 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 66072.8us
W20260709 13:12:54.039976 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 116125us
W20260709 13:12:54.040768 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 48376.7us
I20260709 13:12:54.045674 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.041482 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:54.048342 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:53.948025  2056 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:54.052201 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.021471 29079 rpcz_store.cc:276] 0709 13:12:53.010522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.022559 (+ 12037us) service_pool.cc:224] Handling call
0709 13:12:53.022836 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:53.023047 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:53.023276 (+   229us) log.cc:844] Serialized 14 byte log entry
0709 13:12:53.562015 (+538739us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b18002f2e40. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f4112736e07 00007f411b591ab9 00007f411275369b 00007f411275193f 00007f4112753c02 00007f41153db20b 00007f41153e0d23 00007f41153e123a 00007f411584ba2a
0709 13:12:53.825917 (+263902us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b18002f2e40. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1
0709 13:12:53.827323 (+  1406us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b18002f2e40. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127543f3 00007f41127567c1 00007f4112756779 00007f4112756709 00007f41127566d1 00007f41127559cd 000000000038f354 000000000038f199 00007f411273783a
0709 13:12:53.880805 (+ 53482us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b18002f2e40. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f41192393c7 00007f411584b9a7 00007f411585b0bf 00007f411585b038
0709 13:12:54.010685 (+129880us) log.cc:844] Serialized 14 byte log entry
0709 13:12:54.011061 (+   376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:54.011126 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:54.011193 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.011278 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:54.011413 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34471,"prepare.queue_time_us":201850,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":1459,"raft.queue_time_us":18331,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":5364,"spinlock_wait_cycles":31240832,"thread_start_us":239694,"threads_started":2,"wal-append.queue_time_us":657903}
I20260709 13:12:53.989663  1821 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:54.054231 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 61045.6us
I20260709 13:12:54.059002 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.060443 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.063270 28975 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:54.064493 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.020673  2041 raft_consensus.cc:2804] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:54.069523 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.072847 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:54.070319 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.054167  1906 raft_consensus.cc:697] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:53.990211  1964 consensus_queue.cc:1048] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:12:54.074906 29003 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:54.078255 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.075572 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.079813 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.083292  1906 consensus_queue.cc:237] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.086043  1909 consensus_queue.cc:1048] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:54.090822 28975 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:53.940500  2055 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.066073  1973 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:54.067495  2069 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:54.021853  2053 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:54.110656 28975 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:54.113174 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:54.114187 28904 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51350 (request call id 150) took 41650 ms (client timeout 15000 ms). Trace:
I20260709 13:12:54.125622 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.127863 28975 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:54.097043  1821 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:54.098850  2041 raft_consensus.cc:697] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:54.103576  2037 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:54.072017  1693 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.153759 29009 raft_consensus.cc:1275] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:12:54.161355 28975 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:54.115681  2126 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:54.134716  1897 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:54.144865  2052 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:54.114583 28904 rpcz_store.cc:269] 0709 13:12:12.453417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.453611 (+   194us) service_pool.cc:224] Handling call
0709 13:12:54.103310 (+41649699us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:50.063241 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:50.137732 (+ 74491us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:50.137806 (+    74us) write_op.cc:435] Acquired schema lock
0709 13:12:50.137825 (+    19us) tablet.cc:662] Decoding operations
0709 13:12:50.138287 (+   462us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:50.138384 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:50.138416 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:50.159862 (+ 21446us) tablet.cc:701] Row locks acquired
0709 13:12:50.159915 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:12:50.164983 (+  5068us) write_op.cc:270] Start()
0709 13:12:50.176787 (+ 11804us) write_op.cc:276] Timestamp: P: 1783602770160696 usec, L: 0
0709 13:12:50.176835 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:12:50.222612 (+ 45777us) log.cc:844] Serialized 533 byte log entry
0709 13:12:50.621532 (+398920us) op_driver.cc:464] REPLICATION: finished
0709 13:12:50.718738 (+ 97206us) write_op.cc:301] APPLY: starting
0709 13:12:50.718834 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:50.724847 (+  6013us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:50.768166 (+ 43319us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:50.883237 (+115071us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:51.064420 (+181183us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:12:51.064466 (+    46us) write_op.cc:312] APPLY: finished
0709 13:12:51.064783 (+   317us) log.cc:844] Serialized 25 byte log entry
0709 13:12:51.064979 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:51.065064 (+    85us) write_op.cc:454] Released schema lock
0709 13:12:51.065134 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:51.065717 (+   583us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:51.065744 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6786492,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":96905,"apply.run_cpu_time_us":6059,"apply.run_wall_time_us":346936,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":96833,"prepare.run_cpu_time_us":24679,"prepare.run_wall_time_us":280604,"raft.queue_time_us":130540,"raft.run_cpu_time_us":3967,"raft.run_wall_time_us":35573,"replication_time_us":444459,"thread_start_us":279819,"threads_started":4,"wal-append.queue_time_us":210064,"wal-append.run_cpu_time_us":28779,"wal-append.run_wall_time_us":1634401}]]}
W20260709 13:12:54.234347 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 58232us
I20260709 13:12:54.235463 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.163501  2108 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.240993 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2759) took 1215 ms. Trace:
I20260709 13:12:54.170928  2116 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:54.213606  1973 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.221417  1839 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.249406 29082 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:12:54.256142 29082 pending_rounds.cc:85] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 7
I20260709 13:12:54.256529 29082 pending_rounds.cc:107] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:12:54.256817 29082 raft_consensus.cc:2889] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:54.241439 29007 rpcz_store.cc:276] 0709 13:12:53.025890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.026002 (+   112us) service_pool.cc:224] Handling call
0709 13:12:53.034778 (+  8776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:53.910412 (+875634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:53.910441 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:53.910867 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:12:54.035560 (+124693us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:54.035621 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.036870 (+  1249us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:54.240800 (+203930us) raft_consensus.cc:1685] finished
0709 13:12:54.240821 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:54.240939 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":891169,"raft.queue_time_us":893588,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":207872,"thread_start_us":124288,"threads_started":1,"wal-append.queue_time_us":160558}
I20260709 13:12:54.221666  2056 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:54.245174 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:54.258271 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 102302us
I20260709 13:12:54.280727 28975 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:54.180828  2021 consensus_queue.cc:1048] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:12:54.256074 29048 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.238027  2018 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.278048 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:54.282526 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.283816 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:54.251235  1821 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:54.286796  2019 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:54.244787  2108 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.289289  2108 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:54.293241 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.293320 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:54.294683 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:54.297751 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.298398 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:54.291167 29006 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:54.310093 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 70904.7us
I20260709 13:12:54.300151 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.315918 29048 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.316200 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:54.303355 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.102263 28901 catalog_manager.cc:5697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 7 to 10, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:12:54.323251 29048 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:54.326247 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 45634.2us
I20260709 13:12:54.326649 29048 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.332927 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.325650 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.336740 29009 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:12:54.291945  2052 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:54.345352 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.301883  1906 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:12:54.364257 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 127400us
W20260709 13:12:54.369457 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 56402.5us
I20260709 13:12:54.314940  1771 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:54.258916  1973 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.325146  2012 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:54.380409 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.387704 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:54.388262 29077 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 18.
I20260709 13:12:54.381860 29085 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:12:54.284976  1839 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.382957 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.324648  1917 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:12:54.384470 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 57895.6us
I20260709 13:12:54.336761  2034 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:54.298731  2018 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.393507 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:54.398116  2135 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:12:54.399518  1693 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:54.400904 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:54.401825  1771 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:54.402943 29007 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:12:54.406006 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.382403  2042 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:54.420421  2122 log.cc:927] Time spent T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
I20260709 13:12:54.396377  1906 raft_consensus.cc:2749] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:54.322340  2041 consensus_queue.cc:237] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:54.407102  2053 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:54.429154  2055 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:54.452224 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46343.9us
I20260709 13:12:54.455668 29048 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.434556  1771 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.483696 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.455981  2034 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:54.497334  2138 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:12:54.497862 28901 catalog_manager.cc:5697] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 8 to 11, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 11 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:54.492292 29048 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:54.438181  2042 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:54.487144  2012 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:54.493009  1917 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:54.512639 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.514647 29009 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:54.520579 29155 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11.
I20260709 13:12:54.520282 28901 catalog_manager.cc:5697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:54.502502  1795 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:54.528144 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:54.530900  2132 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
W20260709 13:12:54.533211  2127 log.cc:927] Time spent T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 13:12:54.513837  2123 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:12:54.551517 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.552409 29009 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:54.608358 29005 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
W20260709 13:12:54.539942  2134 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:12:54.712209 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:54.649278 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 97612.1us
I20260709 13:12:54.714203 29122 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:54.539463  1973 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:54.722400  2136 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 13:12:54.543320  1123 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.182s
W20260709 13:12:54.722008  2140 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 13:12:54.609167  2008 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:12:54.730861  2128 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:12:54.763965  1851 consensus_meta.cc:287] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.813s	user 0.009s	sys 0.001s
I20260709 13:12:54.765257  2042 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:54.765329  2047 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:54.779539  2047 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:54.767755  2125 log.cc:927] Time spent T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.556s	user 0.001s	sys 0.000s
I20260709 13:12:54.768342  1839 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:54.719350  2018 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:54.792014  1771 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:54.651692  2133 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:12:54.795233  2088 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:54.795675  2088 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.795997  2088 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:54.797185 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2772) took 1332 ms. Trace:
I20260709 13:12:54.797590 29004 rpcz_store.cc:276] 0709 13:12:53.465006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.470868 (+  5862us) service_pool.cc:224] Handling call
0709 13:12:53.471175 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:53.932098 (+460923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:53.932129 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:53.953360 (+ 21231us) log.cc:844] Serialized 26 byte log entry
0709 13:12:54.163598 (+210238us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:54.163644 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.163708 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:54.796995 (+633287us) raft_consensus.cc:1685] finished
0709 13:12:54.797013 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:54.797131 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":454236,"raft.queue_time_us":474416,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":63,"thread_start_us":108417,"threads_started":1,"wal-append.queue_time_us":311285}
I20260709 13:12:54.807184 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:54.770323  2148 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:12:54.770408  2131 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.232s	user 0.001s	sys 0.001s
W20260709 13:12:54.809666 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 246822us
W20260709 13:12:54.833829  2142 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.339s	user 0.001s	sys 0.001s
W20260709 13:12:54.835453  2137 log.cc:927] Time spent T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.383s	user 0.001s	sys 0.000s
I20260709 13:12:54.836530  2119 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.837278  2119 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:54.768718  2149 consensus_queue.cc:1059] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:12:54.760921  2008 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.831274  1771 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:54.840481  2147 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.849958  2149 raft_consensus.cc:3055] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:54.839998  2119 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:12:54.884807 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 156013us
I20260709 13:12:54.887432  2029 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:54.868284  2149 raft_consensus.cc:740] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:54.863379  2008 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:54.888665 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.893880 29001 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:54.890559 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:54.897074 29005 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:54.871309  2147 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.001003 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:54.870174  1771 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.000403 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:54.893503  2045 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.003893 29150 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:12:55.005275  2149 consensus_queue.cc:260] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.007052 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:55.010260 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:55.012621 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:54.986426  1906 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.027472 29155 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:12:54.887991  2029 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.039600 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.044294 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.045953 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.000571  2145 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:55.028427  2153 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:12:55.058866 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 67049.2us
I20260709 13:12:55.057474 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.060184 29159 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:55.066860  2088 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.068285 29154 raft_consensus.cc:2368] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:55.068331 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:55.030494  2045 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.059446  2153 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.100925 29155 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:55.090464 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.055753  2145 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.052412  1906 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.116704 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.122298 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.104698  2153 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:55.124260 29159 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:12:55.125245 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.119081  1950 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.127419 29159 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:55.130074  2116 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:55.130155 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 854) took 1170 ms. Trace:
I20260709 13:12:55.133152  1906 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.141054 29156 rpcz_store.cc:276] 0709 13:12:53.960010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.960134 (+   124us) service_pool.cc:224] Handling call
0709 13:12:53.986820 (+ 26686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:54.068727 (+ 81907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:54.068766 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:54.177896 (+109130us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b180043e400. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41127222a6 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a
0709 13:12:54.842461 (+664565us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b2c0012c0f8. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411413aa11 00007f411414a372 00007f411587ca01 00007f411584143a 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2
0709 13:12:54.842930 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:12:54.987136 (+144206us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:54.987185 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.987273 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:55.129827 (+142554us) raft_consensus.cc:1685] finished
0709 13:12:55.129847 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.130081 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113826,"raft.queue_time_us":81941,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":6047,"spinlock_wait_cycles":12054656,"thread_start_us":143801,"threads_started":1,"wal-append.queue_time_us":208094}
I20260709 13:12:55.126467  1909 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:55.145509  1909 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.151405 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.152482  2147 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:55.156477  2047 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.128578  2012 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.163859  1906 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
W20260709 13:12:55.164770  2161 log.cc:927] Time spent T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:12:55.177980 29006 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.178478 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:34936 (request call id 2773) took 1692 ms. Trace:
I20260709 13:12:55.178758 29006 rpcz_store.cc:276] 0709 13:12:53.485802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.512633 (+ 26831us) service_pool.cc:224] Handling call
0709 13:12:55.174304 (+1661671us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54000af0d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115838a3d 00007f411a808cd0 00007f4113d8d254 00007f4113d8d1e2 00007f4113d8d111 00007f4113d8d08c 00007f4113d8c342 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:55.178397 (+  4093us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018551168}
I20260709 13:12:55.164690 29153 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:55.167522  2145 leader_election.cc:290] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.167977 28976 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:55.189379 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:55.144732  2116 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:55.171558  2166 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.216235 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:55.215175 28895 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 4 but the committed config has opid_index of 6
W20260709 13:12:55.221958 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:55.144418  1950 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.230762 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.226429  2166 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.188858  1966 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.231747  1771 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.214937  2053 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:55.220517  2037 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:55.174439  2012 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.232851  2088 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:55.175487  1906 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.229375  2145 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:55.261232 29009 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:55.266078 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2787) took 1364 ms. Trace:
I20260709 13:12:55.266598 29002 rpcz_store.cc:276] 0709 13:12:53.901355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.089502 (+188147us) service_pool.cc:224] Handling call
0709 13:12:54.089724 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.136914 (+ 47190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:54.136959 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:54.136999 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:54.137034 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.265762 (+1128728us) spinlock_profiling.cc:243] Waited 47 ms on lock 0x7b5400291654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:55.265830 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.266004 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108100992}
I20260709 13:12:55.243539  1966 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.247247  2008 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.268509  1771 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.304585 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:55.305485 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:55.293390 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:55.307947 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.308065 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:55.308683 29159 raft_consensus.cc:2376] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 11. Re-sending same reply.
W20260709 13:12:55.283293 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 92669us
I20260709 13:12:55.285264  1950 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.292707  2116 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.293179  1906 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.293527  2029 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.294620 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:55.311571 29004 raft_consensus.cc:1813] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Handling vote request from an unknown peer 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:55.312222 29004 raft_consensus.cc:2410] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:55.297058 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:55.312701  2116 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.313268 29005 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:12:55.262524  2145 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.318336 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.323832 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:55.324872  1950 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:55.326058 28975 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:55.327564 28975 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:55.333616 29151 raft_consensus.cc:2393] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 11: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:12:55.336687 28975 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:55.334470  1827 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:55.334600  2175 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:55.337599 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:55.338110 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:55.336869 29122 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:55.347973 29122 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:55.348289 28975 leader_election.cc:400] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15.
I20260709 13:12:55.340283  2029 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:55.310141  2174 consensus_queue.cc:1059] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:12:55.357517  2175 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:55.361438  2145 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.358583  2029 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.341348  1827 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.364328  1827 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:55.363965  2029 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.378268  2008 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:55.379405  2012 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.379944  2012 raft_consensus.cc:493] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:55.380463  2012 raft_consensus.cc:515] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.353987  1972 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:55.389765  1972 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.348393  2166 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.390048  2175 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:55.390203  1972 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:55.317626  1771 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.354818  2147 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:55.390749  2175 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:55.375655  2153 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:55.398736 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 60519.7us
I20260709 13:12:55.399610 29682 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:55.389833  2174 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:55.439913 29155 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:55.395247  2147 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:55.438769 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80556.5us
I20260709 13:12:55.444015  2047 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.444597  2047 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.445731 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:55.445861  2145 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.392234  2180 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:55.401765  2166 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:55.456089 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 106669us
I20260709 13:12:55.457643 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:55.437361  2174 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:55.454380  2132 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:12:55.447036  2147 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:55.474261 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 93184.4us
I20260709 13:12:55.455938  2180 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:55.476388  2037 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:55.480785  2153 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:55.482318  2186 raft_consensus.cc:3055] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:55.446416  2145 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.492869 29048 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:55.471743  2029 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:55.446784  2178 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:12:55.498335 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:55.498966 29083 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:55.502616  2179 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:55.503932 29085 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:12:55.503672  1972 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:55.504319  2181 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:12:55.496093  2029 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.505370  2029 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:55.505864  1972 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:55.510114  1966 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:55.511510  2045 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.516227 29083 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:55.494871 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.461808  2166 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.518266 29078 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
W20260709 13:12:55.492034 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 91439.4us
I20260709 13:12:55.520325  2037 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.520982  2047 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.459378  2053 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:55.493402  2186 raft_consensus.cc:740] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:55.493228  1123 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:55.528075 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.528657 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.530494  2187 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.533244 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.517372  2012 leader_election.cc:290] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.541529 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.524258  2190 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:55.533913  1123 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.565888  2188 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:55.567015  1123 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:55.567981  2186 consensus_queue.cc:260] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.568424 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.560289  2047 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.570072 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:55.570704  2047 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.571693 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.572022  2166 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.572754  2047 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.561169  1771 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.561201  2145 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.573772  2190 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:55.557597  1966 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:55.579041 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.584159 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.585168 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.585886 29152 raft_consensus.cc:2368] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:55.586555 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.586848  1771 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.587491 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.587339  1771 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.588459 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:55.590684 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:55.591238 29078 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:55.563289  1827 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.592414 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.593452 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.575135  2190 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.594324 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.584209 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.598047 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:55.531025  2187 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:55.600312  2046 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:55.593848  1827 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.598747  1771 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.602342  2187 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:55.610710  2187 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:55.616425 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.618089 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.618643 29081 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:55.623216 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.554335 29150 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:55.534832  2149 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:55.525375  2174 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.564081  2180 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:55.563483 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.627907 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:55.875205 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.873282 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.631804 29159 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:55.884847 29007 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:55.883919 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:55.727072 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:55.650758 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.892294 29083 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:55.894402 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:55.610981 29078 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:12:55.898201 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:55.899015 29078 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:55.903054 29155 raft_consensus.cc:1275] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:55.896500 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:55.894738 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.890280 29076 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:12:55.885314 29084 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
W20260709 13:12:55.798771 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 159336us
W20260709 13:12:55.904963 28975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.008s	sys 0.000s
W20260709 13:12:55.905272 28975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.008s	sys 0.000s
W20260709 13:12:55.905552 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 556241us
I20260709 13:12:55.909405 29159 raft_consensus.cc:2393] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 15: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:55.911559 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:55.913692 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.914492 29159 raft_consensus.cc:2410] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:12:55.914361 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:55.915154 29155 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:55.920902 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:55.928916 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:55.929548 29083 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:55.856176  2171 log.cc:927] Time spent T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.409s	user 0.002s	sys 0.001s
I20260709 13:12:55.933241 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:55.933588 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:55.943209 29078 raft_consensus.cc:2468] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 19.
I20260709 13:12:55.957818 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.964418 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.965538 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:55.966360 29076 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:55.794229  2053 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.100862 29155 raft_consensus.cc:2468] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
I20260709 13:12:56.028136 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:56.387472 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:55.991318 29083 raft_consensus.cc:2468] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
W20260709 13:12:56.096102 29048 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.592s (SENT)
I20260709 13:12:56.390666 29048 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:56.391608 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 295580us
W20260709 13:12:56.392443 29048 leader_election.cc:336] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.610s (SENT)
I20260709 13:12:56.392984 29048 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:56.086898 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 88667.3us
I20260709 13:12:55.970988 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:55.927405  2037 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:55.920864  2178 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.401235 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:56.395308 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:56.416213 29007 consensus_meta.cc:287] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Time spent flushing consensus metadata: real 0.527s	user 0.005s	sys 0.004s
I20260709 13:12:56.416630 29007 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:12:55.830331  2190 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:55.640833  2010 raft_consensus.cc:493] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:55.920204  1821 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:55.920500  1950 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:55.933333  2194 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:56.032793  2047 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:56.395347 29079 raft_consensus.cc:1240] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:56.400487  1771 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:56.397372  1973 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:56.426890  1973 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.133716  2008 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:56.405292  2201 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:12:56.405542  2102 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.408475 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:56.366957  2029 consensus_meta.cc:287] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.861s	user 0.008s	sys 0.001s
W20260709 13:12:56.358937  2147 consensus_meta.cc:287] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.867s	user 0.005s	sys 0.003s
I20260709 13:12:56.400012 29050 leader_election.cc:400] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:56.413018  2155 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:56.435920 29050 leader_election.cc:403] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:56.441161 29050 leader_election.cc:304] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:56.414017  2059 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:56.414143 29048 leader_election.cc:336] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.687s (SENT)
I20260709 13:12:56.443181 29048 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:55.959154  1839 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:56.133908  2046 raft_consensus.cc:493] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:56.444998  1771 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:12:56.450736  1795 raft_consensus.cc:2749] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:56.451460  1795 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:56.452288  1795 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:56.453184  2186 raft_consensus.cc:3060] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:56.448843  1839 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.455582  1839 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.455623 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:56.456367  2155 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:56.456722  2155 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:56.441844  2147 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.457496  2155 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.457983 29154 raft_consensus.cc:2368] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:56.440155  2102 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.460357  2102 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.436597  2029 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.461400  2205 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:56.462989 29156 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 17: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:56.463282  2046 raft_consensus.cc:515] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.463822  2029 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:56.464854  1973 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.420193  2037 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.422921  2178 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.476315 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:56.478088 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:56.428221  2190 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.424623  2010 raft_consensus.cc:515] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.482235 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:12:56.429996  1821 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.435947  1950 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.443342  2008 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.475240  2198 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:56.482908  2102 raft_consensus.cc:2749] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:56.493750  1982 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:56.494411  1982 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:56.498008  1982 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:56.502868  2149 consensus_meta.cc:287] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Time spent flushing consensus metadata: real 0.575s	user 0.004s	sys 0.005s
I20260709 13:12:56.509507  2188 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:56.510733 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:56.512085  2046 leader_election.cc:290] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.514209 29153 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:56.521896  2147 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:56.528076  2008 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.428398  2194 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.531131  2194 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.448163  2047 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:56.515825 29153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35272 (request call id 2083) took 1211 ms (client timeout 1516 ms). Trace:
W20260709 13:12:56.546181 29153 rpcz_store.cc:269] 0709 13:12:55.304163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.304306 (+   143us) service_pool.cc:224] Handling call
0709 13:12:56.515630 (+1211324us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540013e554. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:56.515759 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2755390720}
I20260709 13:12:56.498799  2178 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.546916 29151 raft_consensus.cc:2468] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:56.449946  2059 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.531795  2194 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:56.547590  1827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.045s	sys 0.043s
I20260709 13:12:56.551867  2037 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.603168  2189 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:56.603742  2189 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:12:56.590356  1827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.045s	sys 0.043s
I20260709 13:12:56.613445  2178 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:56.597155  2194 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:56.691080 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 276970us
W20260709 13:12:56.688643 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 244601us
W20260709 13:12:56.695721 29048 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.716s (SENT)
I20260709 13:12:56.697564 29048 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:56.698720 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Timed out: RequestConsensusVote RPC to 127.27.63.65:43941 timed out after 1.822s (SENT)
I20260709 13:12:56.699153 29048 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:56.714762 29077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2501) took 1056 ms. Trace:
W20260709 13:12:56.716467 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 782865us
I20260709 13:12:56.715418 29077 rpcz_store.cc:276] 0709 13:12:55.658196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.658376 (+   180us) service_pool.cc:224] Handling call
0709 13:12:55.658761 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.659001 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:56.323660 (+664659us) log.cc:844] Serialized 14 byte log entry
0709 13:12:56.714312 (+390652us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.714383 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:56.714446 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.714538 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.714687 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4381,"prepare.queue_time_us":813129,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":7977,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":865031,"threads_started":3,"wal-append.queue_time_us":300620}
I20260709 13:12:56.720433 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:56.690564  2037 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:56.723199 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 297485us
I20260709 13:12:56.728813 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:56.729787 29153 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:12:56.730058 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:56.734455 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:56.734843 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:56.737452 29125 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:56.738471 29155 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:56.688406  2178 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.739567 29682 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:56.732652 29004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2828) took 1104 ms. Trace:
I20260709 13:12:56.741329 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:56.743044 29158 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:56.743214 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:56.744015 29004 rpcz_store.cc:276] 0709 13:12:55.629591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.630030 (+   439us) service_pool.cc:224] Handling call
0709 13:12:55.896168 (+266138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.896398 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:55.931855 (+ 35457us) log.cc:844] Serialized 14 byte log entry
0709 13:12:56.460677 (+528822us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b18003fcca0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f411b61cde1 00007f411b6295b1 00007f41126cec0a 00007f411be0e219 00007f4112756a40 00007f41127514e4 00007f4112751455 00007f4115839060 00007f411584b716 00007f411585c287 00007f411585c208
0709 13:12:56.732255 (+271578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.732324 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:56.732382 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.732461 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.732588 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16304,"prepare.queue_time_us":37424,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":559619,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":35181184,"thread_start_us":266291,"threads_started":1}
I20260709 13:12:56.748996 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:56.750334 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51664.4us
I20260709 13:12:56.750516 29004 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:56.751107 29125 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:56.752439 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:56.753566  1969 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:56.754438  1969 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:56.754702  2153 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:56.755571  2047 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.755652  1969 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.689217  2189 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.758677  2010 leader_election.cc:290] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.759290  1950 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.761018 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:56.753722  1964 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:56.766594 29050 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:56.768184 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:56.762982 29682 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:56.770195 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:56.775315 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:56.779700 29714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49414 (request call id 793) took 1213 ms. Trace:
I20260709 13:12:56.780354 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:56.780089 29714 rpcz_store.cc:276] 0709 13:12:55.566523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.573113 (+  6590us) service_pool.cc:224] Handling call
0709 13:12:55.573466 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.601397 (+ 27931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:55.601439 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:55.601486 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:55.601524 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.779494 (+1177970us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b54005b8454. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:56.779542 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.779652 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63797632}
I20260709 13:12:56.797940 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:56.800509 29004 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:56.801316 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:56.804946 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:56.798017 29125 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:56.727792  2037 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.807063 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:56.807052 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:56.751055  2019 consensus_queue.cc:1048] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:12:56.778678  2149 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:56.779052  2213 raft_consensus.cc:2804] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:56.778515  2206 raft_consensus.cc:2804] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:56.786702  2165 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:12:56.792500  2153 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:56.822141 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:56.794988  2190 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.829532 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:56.802163  1851 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:56.838421 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2825) took 1211 ms. Trace:
I20260709 13:12:56.838925 29010 rpcz_store.cc:276] 0709 13:12:55.627210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.668517 (+ 41307us) service_pool.cc:224] Handling call
0709 13:12:56.837543 (+1169026us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:56.837778 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.837986 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.838022 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.838089 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:56.838134 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.838203 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.838353 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16461056}
I20260709 13:12:56.830953 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:56.833149  2210 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:12:56.833442  2037 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.804056  2208 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:12:56.803722  2216 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:56.793821  2221 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:56.880228 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 74476.1us
I20260709 13:12:56.828927  2153 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.815680  2053 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:56.891209 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46574.3us
I20260709 13:12:56.814390  1821 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.825651  1966 raft_consensus.cc:2804] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:56.816116  2029 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:56.806915  2220 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:56.900645 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 81083.5us
W20260709 13:12:56.901171 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.901552 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:56.830082  1771 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:56.880748  2205 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.892741  2106 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.928506  2106 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.930433  2194 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.931042  2194 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:56.931375  2106 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:56.931524  2194 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:56.932647  2194 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.934967  2232 raft_consensus.cc:476] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Not starting pre-election -- already a leader
I20260709 13:12:56.935609  2232 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:56.935967  2232 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:56.936391  2232 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:56.937666  2220 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.945120 29009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2824) took 1318 ms. Trace:
I20260709 13:12:56.826363  2222 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:56.945597 29009 rpcz_store.cc:276] 0709 13:12:55.626940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.629562 (+  2622us) service_pool.cc:224] Handling call
0709 13:12:56.944351 (+1314789us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:56.944573 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.944752 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.944786 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.944828 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:56.944873 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.944932 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.945040 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12916992}
I20260709 13:12:56.947135 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:56.948832  2220 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.950603  1906 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:56.951354  1906 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.954396  1906 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.955552 29083 raft_consensus.cc:1240] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:56.957706 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:56.958597 29077 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:56.959981 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:56.899480  2217 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:56.891466  2198 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:56.918000 29683 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:56.921746 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 125413us
I20260709 13:12:56.906883  2153 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:56.924299 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2829) took 1294 ms. Trace:
I20260709 13:12:56.907313  2221 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:56.926198  2178 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:56.926597  1950 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:56.919380  2029 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:56.921713  1966 raft_consensus.cc:697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:56.968562 29080 raft_consensus.cc:1240] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:12:57.017558 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 99675us
I20260709 13:12:57.018610 29006 rpcz_store.cc:276] 0709 13:12:55.629882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.630099 (+   217us) service_pool.cc:224] Handling call
0709 13:12:56.923569 (+1293470us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:56.923784 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.923974 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.923996 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.924056 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:56.924088 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.924140 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.924244 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12290944}
I20260709 13:12:56.960104 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:56.961468 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:56.891160  2218 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:56.917856 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.024574 29010 raft_consensus.cc:2468] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
W20260709 13:12:57.023924 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 76753.9us
I20260709 13:12:56.966879 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.021724 29050 leader_election.cc:400] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:56.964447 29001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 991) took 1078 ms. Trace:
I20260709 13:12:57.026266 29050 leader_election.cc:403] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:57.026333 29001 rpcz_store.cc:276] 0709 13:12:55.885642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.406018 (+520376us) service_pool.cc:224] Handling call
0709 13:12:56.406464 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:56.689474 (+283010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.689519 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:56.701367 (+ 11848us) log.cc:844] Serialized 26 byte log entry
0709 13:12:56.884572 (+183205us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:56.884625 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.884716 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:56.964160 (+ 79444us) raft_consensus.cc:1685] finished
0709 13:12:56.964189 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.964369 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67608,"raft.queue_time_us":145569,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":30562,"thread_start_us":321031,"threads_started":2,"wal-append.queue_time_us":162426}
I20260709 13:12:56.844874  2206 raft_consensus.cc:697] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:56.845824  2213 raft_consensus.cc:697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 19 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:56.919066  2228 raft_consensus.cc:493] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.041160 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:57.041499 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:57.042094 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.042845 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:57.056784 29081 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:57.045616 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:57.058877 29082 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:12:57.046388 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.045253 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.047142 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:57.103854 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 274387us
W20260709 13:12:57.105837 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 65503.2us
I20260709 13:12:57.025910  1995 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:56.957290  2230 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:12:57.108352 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71195.6us
I20260709 13:12:57.053213 29154 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:57.109688  1966 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.110172  2029 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.112974  1909 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.035918  1950 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.036172  2233 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:12:57.115011  2037 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:57.128227 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:56.968887  1821 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.155s
I20260709 13:12:57.136538 29081 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:12:57.139585 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.125342  2059 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.127681  2174 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:57.142426  2214 log.cc:927] Time spent T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
W20260709 13:12:57.155588 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 253801us
I20260709 13:12:57.035346  2198 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.036432  2218 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:57.159405 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.152776  2230 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:57.158149  2174 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.162497 29081 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:57.163527  2221 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.156411  2213 consensus_queue.cc:237] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.163789 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.165103 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.165939 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.166203 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.169162  1950 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.102677  2228 raft_consensus.cc:515] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.178874  2232 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.186749 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.192723 29083 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:57.188936 28976 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:57.186429 29005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 988) took 1302 ms. Trace:
I20260709 13:12:57.191957 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.192135 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.194397 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.194970 29010 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:12:57.193583 29005 rpcz_store.cc:276] 0709 13:12:55.884478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.892299 (+  7821us) service_pool.cc:224] Handling call
0709 13:12:57.185769 (+1293470us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b5800000c98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:57.185950 (+   181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.186129 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:57.186155 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:57.186195 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:12:57.186230 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.186284 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.186376 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9542528}
I20260709 13:12:57.196128 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:57.196276 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.197021 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:57.197053 29010 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:12:57.198251  2221 consensus_queue.cc:1059] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:57.198758  2240 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:57.199621 29159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2789) took 1233 ms. Trace:
I20260709 13:12:57.200007 29159 rpcz_store.cc:276] 0709 13:12:55.966177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.409051 (+442874us) service_pool.cc:224] Handling call
0709 13:12:56.409396 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.495148 (+ 85752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.495189 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.495243 (+    54us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:12:56.495297 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.199408 (+704111us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b54003f66d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.199454 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.199571 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196611968}
I20260709 13:12:57.197319 29125 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:57.202015 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:57.203329 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.145601  2037 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.180601  2230 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.206851  2230 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:57.213402 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:57.213547 29005 raft_consensus.cc:2468] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:12:57.214862 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.215662 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.218278  2232 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.191061  2206 consensus_queue.cc:237] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:57.219749 29003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34936 (request call id 2820) took 1929 ms (client timeout 2223 ms). Trace:
I20260709 13:12:57.212234  2238 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:57.220798  2153 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:57.222204 29003 rpcz_store.cc:269] 0709 13:12:55.290387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.293171 (+  2784us) service_pool.cc:224] Handling call
0709 13:12:57.219561 (+1926390us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540022c254. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:57.219697 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3241612544}
I20260709 13:12:57.224864 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:57.229864 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.230468 29048 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.231014 29080 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:57.216245  2194 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:57.226637 29156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2785) took 1351 ms. Trace:
I20260709 13:12:57.233572 29156 rpcz_store.cc:276] 0709 13:12:55.874818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.874965 (+   147us) service_pool.cc:224] Handling call
0709 13:12:57.226403 (+1351438us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54003106d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:57.226581 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329080704}
I20260709 13:12:57.233847  1909 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:57.234018  1906 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:57.234457  1909 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.234954  1909 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:57.236506  2155 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:57.237839 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.238101  2037 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.238974 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:57.232648  2232 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:57.240402  2232 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:57.240759 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.227377 29083 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:12:57.228181 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:57.240777  2232 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:57.242388 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.239971 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.243067 29009 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:12:57.243376 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.243839 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.226838 29079 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:57.232817 29048 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.246243 29048 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.247046 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.240870  2238 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:57.249993 29048 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.228578 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.247074 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.232108 29010 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:12:57.251425 29005 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:12:57.251579 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.252272 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.252631 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.254292 29005 raft_consensus.cc:2393] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 17: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:57.259718 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.260352 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.261125 29048 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.261123 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.262827 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
W20260709 13:12:57.264499  2243 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:12:57.264669 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:57.265452 29151 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35260 (request call id 2779) took 1709 ms (client timeout 1577 ms). Trace:
I20260709 13:12:57.265699 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:57.265846 29151 rpcz_store.cc:269] 0709 13:12:55.556027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.563214 (+  7187us) service_pool.cc:224] Handling call
0709 13:12:57.265234 (+1702020us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b540013c754. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158452fa 00007f411a806b17 00007f4113d816d4 00007f4113d81662 00007f4113d81591 00007f4113d8150c 00007f4113d807c2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:57.265339 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1474839296}
I20260709 13:12:57.266265 29708 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:12:57.266652  1995 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:57.267177  1995 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.267889  2237 raft_consensus.cc:2749] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:57.268811 29717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47384 (request call id 1016) took 1406 ms. Trace:
I20260709 13:12:57.269445  2174 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.269495  1995 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:57.270872 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.271879  2228 leader_election.cc:290] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.272272 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.272451  2229 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:57.253377  1906 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.270903 29157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35260 (request call id 2790) took 1298 ms. Trace:
I20260709 13:12:57.258610  1821 consensus_queue.cc:1059] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:12:57.273967 29157 rpcz_store.cc:276] 0709 13:12:55.972951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.417625 (+444674us) service_pool.cc:224] Handling call
0709 13:12:56.417888 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.454925 (+ 37037us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:56.718887 (+263962us) log.cc:844] Serialized 14 byte log entry
0709 13:12:56.783332 (+ 64445us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.783409 (+    77us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:56.783477 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.270704 (+487227us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b54004010d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.270749 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.270852 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4985,"prepare.queue_time_us":17981,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":1112,"raft.queue_time_us":10504,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":10304,"spinlock_wait_cycles":84645248,"thread_start_us":326586,"threads_started":2,"wal-append.queue_time_us":436049}
I20260709 13:12:57.275408  1821 raft_consensus.cc:3055] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:57.275804  1821 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:57.276189 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.276613  1821 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.277786  1821 raft_consensus.cc:3060] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:57.256637  2217 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:57.280275 29125 leader_election.cc:304] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.268023  2188 raft_consensus.cc:3055] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:57.280229 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.269155 29717 rpcz_store.cc:276] 0709 13:12:55.862766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.862935 (+   169us) service_pool.cc:224] Handling call
0709 13:12:55.892932 (+ 29997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.548633 (+655701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.548659 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.548697 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:56.548736 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.268624 (+719888us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54004b69d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.268660 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.268762 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507422592}
I20260709 13:12:57.281878 29005 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:57.283828 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.288631 29005 raft_consensus.cc:1275] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:57.288801 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.297392 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.297828 29048 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.301370  1906 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:57.301863  1906 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:57.302237  1906 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:57.307719 29125 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.311610 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.313999  2106 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:57.315680 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.316859 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.284720  2188 raft_consensus.cc:740] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:57.255236  2037 consensus_queue.cc:1059] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:12:57.319181  2217 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.384614  2238 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.392660  1909 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.395464  1909 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.312985  2008 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.397409  2153 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:57.312992  2229 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:57.365770 29048 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.305239  2116 raft_consensus.cc:2749] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:57.398224  2194 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:57.401149  2232 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.258981  2046 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:12:57.322721  2106 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:57.414172 29048 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.379750 29082 raft_consensus.cc:2468] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:57.404630  2229 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:57.415812 29081 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:57.411772  2194 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.367970  2228 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:57.417284  2239 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:12:57.417348  2217 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:57.419816  1966 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.425988 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.413688  1909 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.422811 29005 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:57.417513  2229 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:57.436656  2232 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.432919  2155 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.450312  2155 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.404198  2188 consensus_queue.cc:260] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.437052  2030 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.437332 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:57.453151 29082 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:57.431233  1771 consensus_queue.cc:1048] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:12:57.429941  1966 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.457865 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.458318 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:57.458600 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68c287f2818e4a23b258f6c19741c26a" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.439884  2205 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:57.460110 29009 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 11.
I20260709 13:12:57.461958 29082 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:57.462440 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:57.464566 29003 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:12:57.441512  2194 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.429049  2216 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:57.464637 28976 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:57.488276 29009 raft_consensus.cc:1240] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:57.455487  1906 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.484825  2205 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:57.526332 29006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2840) took 1090 ms. Trace:
I20260709 13:12:57.526892 29006 rpcz_store.cc:276] 0709 13:12:56.436078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.019950 (+583872us) service_pool.cc:224] Handling call
0709 13:12:57.020236 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.153458 (+133222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:57.153488 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:57.153523 (+    35us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:57.153550 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.526018 (+372468us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b540035d9d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.526088 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.526259 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305889024}
W20260709 13:12:57.530222 28974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.535961 29007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2839) took 1110 ms. Trace:
I20260709 13:12:57.536504 29007 rpcz_store.cc:276] 0709 13:12:56.429884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.684033 (+254149us) service_pool.cc:224] Handling call
0709 13:12:56.684377 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.779637 (+ 95260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.779669 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.779707 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:56.779735 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.535683 (+755948us) spinlock_profiling.cc:243] Waited 95.1 ms on lock 0x7b5400291654. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.535744 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.535892 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218629504}
I20260709 13:12:57.548152 29155 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:57.548615 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:57.551601 29076 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44428 (request call id 2507) took 1682 ms. Trace:
I20260709 13:12:57.555399 29008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 993) took 1669 ms. Trace:
I20260709 13:12:57.555796 29008 rpcz_store.cc:276] 0709 13:12:55.886289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.437818 (+551529us) service_pool.cc:224] Handling call
0709 13:12:56.438192 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.548171 (+109979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.548202 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.548248 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:56.548285 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.555217 (+1006932us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54002279d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.555257 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.555351 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252355840}
I20260709 13:12:57.547709 29002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 2838) took 1125 ms. Trace:
I20260709 13:12:57.559115 29002 rpcz_store.cc:276] 0709 13:12:56.422146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.544654 (+122508us) service_pool.cc:224] Handling call
0709 13:12:56.544907 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.719687 (+174780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.719733 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.719790 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:56.719846 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.547509 (+827663us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54003e26d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.547551 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.547656 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401451264}
I20260709 13:12:57.560034 29007 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:57.552129 29076 rpcz_store.cc:276] 0709 13:12:55.869643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.967308 (+ 97665us) service_pool.cc:224] Handling call
0709 13:12:57.550740 (+1583432us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b5800004b98. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271cb71 00007f411271ca22 00007f411b635c09 00007f411a78fee0 00007f411a8590ab 00007f411a859131 00007f411a81afc0 00007f411a805929 00007f411a8053d2 00007f4113d7cbd4
0709 13:12:57.550972 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.551201 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:57.551233 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:57.551287 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:57.551341 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.551412 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.551537 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2384128}
W20260709 13:12:57.565860 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.568332 29085 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:57.568361 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.570405 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:57.584801 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.491343  2216 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:57.592877 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 164517us
I20260709 13:12:57.605233 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.607685 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:57.608450 29004 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:57.606845 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.614779 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.618372 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:57.619226 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 110100us
I20260709 13:12:57.578115  2205 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:57.586427  2228 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:57.579895  1771 raft_consensus.cc:493] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.619771  2155 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.629547 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.630029  1906 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:57.630570  2249 log.cc:927] Time spent T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:12:57.630228  2205 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.631945 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.634428 29715 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:57.582882  2176 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.636716 29083 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:12:57.645485 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:57.646664 29048 leader_election.cc:304] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.651388 29048 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.652387  2155 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:57.650866  1906 raft_consensus.cc:2749] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:57.657577  2216 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.651043  2176 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.661053  2176 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.637694  1771 raft_consensus.cc:515] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.644490  2228 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.663297 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.663730 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.665109  2228 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.661995 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.667117 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:57.668757 29006 raft_consensus.cc:2410] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:12:57.671319 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:57.649806  2200 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 1.258s	user 0.006s	sys 0.000s
I20260709 13:12:57.635515  2230 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.693408 29004 raft_consensus.cc:2468] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:12:57.700948  2229 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.707433  1771 leader_election.cc:290] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.708465 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.708886 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d989a55ed204649a0440142ce55e5ef" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:57.711062  2253 log.cc:927] Time spent T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:12:57.720860  2230 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.722204 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:57.743678 29682 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.732898 29150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32836 (request call id 864) took 1831 ms. Trace:
I20260709 13:12:57.750716 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:57.751331 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94045d5a8709418e82919c9fbf804582" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:57.753159 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:57.746217 29150 rpcz_store.cc:276] 0709 13:12:55.901590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.901775 (+   185us) service_pool.cc:224] Handling call
0709 13:12:57.272331 (+1370556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.272546 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:57.272831 (+   285us) log.cc:844] Serialized 14 byte log entry
0709 13:12:57.434582 (+161751us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:57.434638 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:57.434681 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.434759 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.732687 (+297928us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540013c750. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:57.732827 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22210,"prepare.queue_time_us":573,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":22630,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":3080125824,"thread_start_us":139179,"threads_started":1,"wal-append.queue_time_us":145745}
I20260709 13:12:57.736931  2021 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:57.766212 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:57.768486 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:57.749732  2190 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:57.763402  2021 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:57.782161 29050 leader_election.cc:304] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.790289 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:57.754202  2002 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:12:57.796695 28975 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:57.798785 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:57.802099  1966 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.803323  2218 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.808537 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.809795  2174 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:57.810329  2188 raft_consensus.cc:3060] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:57.811678  2229 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:57.792300  2158 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.817405  2158 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.821820 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:57.790702  2255 raft_consensus.cc:2749] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.820448  2053 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:57.836304 29715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47388 (request call id 638) took 1035 ms. Trace:
I20260709 13:12:57.836694 29715 rpcz_store.cc:276] 0709 13:12:56.800520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.800663 (+   143us) service_pool.cc:224] Handling call
0709 13:12:56.811441 (+ 10778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.643186 (+831745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.836096 (+192910us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54006b53d4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.836255 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892195712}
I20260709 13:12:57.839655 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:57.842782 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:57.844848 29051 leader_election.cc:304] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.845101 29079 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:57.845877  2188 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:57.846010 29051 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.846292  2188 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:57.847532  2228 raft_consensus.cc:2749] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:57.849556 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.849694  2053 consensus_queue.cc:1059] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:57.850419  2053 consensus_queue.cc:1243] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer ca45420e033847489ad989c1c6eed3ef log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:12:57.850646 29048 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.847600  2254 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.852396 29048 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.857405  2233 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:57.858290  1771 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.858595 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.862027  2198 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.864643 29003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 998) took 1160 ms. Trace:
I20260709 13:12:57.865036 29003 rpcz_store.cc:276] 0709 13:12:56.703782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.253919 (+550137us) service_pool.cc:224] Handling call
0709 13:12:57.464122 (+210203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.465211 (+  1089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.864474 (+399263us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54003f0cd0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41158408ad 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6 00007f4113b7b26f
0709 13:12:57.864590 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482826496}
I20260709 13:12:57.855585  2194 raft_consensus.cc:3055] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:57.866429  2194 raft_consensus.cc:740] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:57.867179  2194 consensus_queue.cc:260] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.868419  2194 raft_consensus.cc:3060] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:57.856689 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:57.844887  2189 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:57.875877 29009 raft_consensus.cc:1240] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:57.853478 29084 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:12:57.852198 29051 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.878683  2237 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:57.880836  2195 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:57.884660  2158 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:57.886229  2195 consensus_queue.cc:1059] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:57.863687 29122 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.887437  2195 raft_consensus.cc:3055] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:57.887487  2053 consensus_queue.cc:1059] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:57.887903 29125 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:57.895506  1795 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:57.895875 29048 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:57.896257  1973 raft_consensus.cc:2804] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:57.896679  1795 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:57.896823  1973 raft_consensus.cc:697] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:57.897161 29048 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.897979  1795 raft_consensus.cc:2804] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:57.897758  1973 consensus_queue.cc:237] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:57.898432 29048 leader_election.cc:304] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.898466  1795 raft_consensus.cc:697] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:57.899456  1795 consensus_queue.cc:237] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.887969  2195 raft_consensus.cc:740] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:57.900866 29003 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:12:57.901234 29003 pending_rounds.cc:85] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 6
I20260709 13:12:57.901608 29003 pending_rounds.cc:107] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:12:57.901157  2195 consensus_queue.cc:260] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.901978 29003 raft_consensus.cc:2889] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:57.902594  2195 raft_consensus.cc:3060] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:57.901638 29005 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:12:57.912714 29005 pending_rounds.cc:85] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 9
I20260709 13:12:57.912966 29005 pending_rounds.cc:107] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:12:57.913197 29005 raft_consensus.cc:2889] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:57.903966 29122 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.890080 29010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33750 (request call id 1001) took 1174 ms. Trace:
I20260709 13:12:57.914706 29010 rpcz_store.cc:276] 0709 13:12:56.715713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.256484 (+540771us) service_pool.cc:224] Handling call
0709 13:12:57.256704 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.421051 (+164347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:57.421093 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:57.421134 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:57.421167 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.889882 (+468715us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540035ded4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f4115841e01 00007f411584083e 00007f411a805419 00007f4113d7cbd4 00007f4113d7cb62 00007f4113d7ca91 00007f4113d7ca0c 00007f4113d7bcc2 00007f4113b7c48c 00007f4113b7b8c6
0709 13:12:57.889924 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.890029 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377527680}
I20260709 13:12:57.909192 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:57.917629 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:57.917538 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:57.920225 29084 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 12. Current term is 13. Ops: []
W20260709 13:12:57.924163 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 67572.1us
I20260709 13:12:57.926321 29076 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:12:57.926676 29076 pending_rounds.cc:85] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Aborting all ops after (but not including) 11
I20260709 13:12:57.926877 29076 pending_rounds.cc:107] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:12:57.927098 29076 raft_consensus.cc:2889] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:57.928541 29078 raft_consensus.cc:1275] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:12:57.855666  2188 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:57.933563 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:57.934832 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:57.891885  1851 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:57.999265  2021 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:57.931480  1821 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:58.000545  1821 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:58.001230 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:58.000972  1821 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:57.917162  2232 raft_consensus.cc:3055] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:57.912241  2237 raft_consensus.cc:2749] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:58.005599  2232 raft_consensus.cc:740] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:57.999269  1851 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.020961  2238 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:58.021386  2238 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:58.005488  2174 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:12:58.022007  2238 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:58.022388  1851 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:57.994760 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 62529.1us
I20260709 13:12:57.916680  2046 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:57.943830  1123 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:57.999970 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:58.033694 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:58.020135  2232 consensus_queue.cc:260] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:58.027771  2243 log.cc:927] Time spent T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.097s	user 0.000s	sys 0.007s
I20260709 13:12:58.031292 29076 raft_consensus.cc:1240] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:58.032437 29084 raft_consensus.cc:1240] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:58.034683 29050 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.037420 29080 raft_consensus.cc:1813] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Handling vote request from an unknown peer 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:58.050444 29080 raft_consensus.cc:2410] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:58.033598 29085 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:12:58.051487 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:58.004010 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 70510.4us
I20260709 13:12:58.041101 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
W20260709 13:12:57.998389 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 209987us
I20260709 13:12:58.044169 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:58.044235 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.053896 29122 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.029412  1839 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.064118 29683 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.066371 29079 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:12:58.057413 29157 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:58.036072  2174 consensus_queue.cc:1059] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:12:58.072695 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.075438 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.075832 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.072722 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.055737  2237 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:58.068989  1839 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.089941  1839 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:58.064709  2232 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:58.096618 29002 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:12:58.097029 29002 pending_rounds.cc:85] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 11
I20260709 13:12:58.097263 29002 pending_rounds.cc:107] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:12:58.097496 29002 raft_consensus.cc:2889] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:58.099793  2180 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.069821  1151 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:58.100373  2180 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:58.073633  2202 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:58.084714  1966 consensus_queue.cc:1059] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:58.091939 29154 raft_consensus.cc:1240] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:58.077494 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:58.129226 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.129868 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 80993.5us
W20260709 13:12:58.130949 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 70647.8us
I20260709 13:12:58.134047 29157 raft_consensus.cc:1240] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:58.135131 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.135594 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:58.137451 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.138818 29159 raft_consensus.cc:1240] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:58.139519 28975 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.140632 29152 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:12:58.142359 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.143839 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:58.145706 29158 raft_consensus.cc:1240] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 19 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 13:12:58.146780 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.147768  2029 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:58.148742  2029 raft_consensus.cc:3055] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:58.149098  2029 raft_consensus.cc:740] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:58.149509  1950 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:58.150341  1950 consensus_queue.cc:1059] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:58.151340  1950 raft_consensus.cc:3055] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:58.148567 28975 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.160872 28975 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.161936 28975 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.151738  1950 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:58.149930  2029 consensus_queue.cc:260] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:58.168869  1123 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:58.119026  2202 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:58.172775 28975 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.177229 29157 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:58.180321 28975 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.188434 28975 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:57.218444 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:58.183796 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.171631  2174 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:58.178960  2205 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:58.182123  2153 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.182046  2198 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:58.176344  2189 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:58.206490  1982 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:58.176671  1123 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:58.208688  1123 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:58.209775  1123 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:58.199429  2153 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.214007  2153 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:58.214555  2153 raft_consensus.cc:3055] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:58.214892  2153 raft_consensus.cc:740] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:58.216120  2238 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.184717  2029 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:58.218573  2238 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:58.179247  2188 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.220669  2188 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:58.173380  1821 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:58.229141  1821 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:58.229806  2180 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:58.239490  2053 raft_consensus.cc:3055] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:58.240017  2053 raft_consensus.cc:740] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:12:58.215617  2153 consensus_queue.cc:260] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:58.245030 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 56675us
I20260709 13:12:58.247141 28975 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.247682  2180 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:58.189230  1839 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:12:58.249295  1821 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:58.193532  2278 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:12:58.253242 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
W20260709 13:12:58.250609 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.255692  2021 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:58.262693 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.262952 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:58.191720  2037 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:58.278177 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:12:58.278369 29009 raft_consensus.cc:1240] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:58.256179  1821 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:58.283123 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:58.283639 29079 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:12:58.283211  2239 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.324682 29153 raft_consensus.cc:1240] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:58.339841  2239 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:58.361021 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 172224us
W20260709 13:12:58.363216 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 174089us
I20260709 13:12:58.323586  2237 consensus_queue.cc:1059] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:12:58.380661  1839 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:12:58.367594  1851 consensus_queue.cc:1059] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:12:58.391741  2251 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.459s	user 0.010s	sys 0.000s
W20260709 13:12:58.443820 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60045.6us
W20260709 13:12:58.443938 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60128.3us
I20260709 13:12:58.443910  1123 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:58.448946 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.398532  1795 raft_consensus.cc:3055] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:58.446105 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:58.388006  2239 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:58.451742 29009 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
I20260709 13:12:58.451495  1795 raft_consensus.cc:740] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:12:58.490938 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:58.474910 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:12:58.491451 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:12:58.501187 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.504055 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:58.467787  2046 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:58.503685 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 50060.6us
I20260709 13:12:58.496035  2195 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:12:58.500699  2254 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:58.514838  1821 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:58.535403  1795 consensus_queue.cc:260] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.540827 29082 raft_consensus.cc:1240] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:58.538975  2254 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:58.569711 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 50749.9us
I20260709 13:12:58.526032  2195 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:58.572700 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 60508.3us
I20260709 13:12:58.573679 29050 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:58.576393 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.565475  2254 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:58.566794  1795 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:58.581493  1151 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:58.578689  2239 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:58.583817  1969 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:58.588866  1966 consensus_queue.cc:1059] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:58.590303  1966 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:58.592506  2147 consensus_queue.cc:1048] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:58.593966 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:58.595026 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:58.595818 29010 raft_consensus.cc:2410] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:12:58.596599 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:58.604681 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:58.618781  2153 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:58.620525  2106 raft_consensus.cc:3055] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:58.620966  2106 raft_consensus.cc:740] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:58.620412  2254 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:58.621691  2106 consensus_queue.cc:260] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:58.622951  2106 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:58.624277 29079 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:12:58.624557  2254 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:58.638306 29085 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:58.643467 29711 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:58.644945  2195 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:58.646129  2284 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:12:58.646036 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:58.648252 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:58.664834  2269 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
W20260709 13:12:58.666435 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 83195.4us
I20260709 13:12:58.665540  2269 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.678735 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:58.680119 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:12:58.682505 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.689864  2198 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:58.689498  2053 consensus_queue.cc:260] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:58.690403  2198 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.690800  2053 raft_consensus.cc:3060] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:58.692160  2269 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:58.693099 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:58.693276  2198 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:12:58.694340 28976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.711848 29710 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:58.717530 29004 raft_consensus.cc:1275] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:12:58.717938 29004 pending_rounds.cc:85] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 10
I20260709 13:12:58.718153 29004 pending_rounds.cc:107] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:12:58.718369 29004 raft_consensus.cc:2889] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:58.718459  1950 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:58.721553 28976 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.722046  2217 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:58.722666 28976 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.723687  2053 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:58.724285  1821 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:58.724375  2053 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:58.725235  2285 consensus_queue.cc:1059] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:58.726205  2285 raft_consensus.cc:3055] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:58.726531  2285 raft_consensus.cc:740] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:58.726846 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.727444 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:58.727516  2285 consensus_queue.cc:260] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:58.728487  2174 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:58.729008  2285 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:58.729596 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.731226  2053 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.739413  1950 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:58.744932 29001 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:12:58.745500 29001 pending_rounds.cc:85] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 8
I20260709 13:12:58.745806 29001 pending_rounds.cc:107] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:12:58.746140 29001 raft_consensus.cc:2889] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:58.750818  2263 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:58.786443 29004 raft_consensus.cc:1275] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:58.787703  2239 consensus_queue.cc:1048] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:58.800755 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.805380 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 128195us
I20260709 13:12:58.822606 29009 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:12:58.856544 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 50277.3us
I20260709 13:12:58.859118  1123 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:58.868183 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.878091 29154 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:58.890448 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:12:58.895431 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:58.896653 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:12:58.897392 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 83112.4us
I20260709 13:12:58.898874 29154 raft_consensus.cc:1275] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:12:58.900643 29158 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:12:58.898270 29122 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.909605 29150 raft_consensus.cc:1240] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:12:58.917547 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:58.911882 29682 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:58.931504 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:58.937338 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:12:58.920256 28902 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 7 to 12, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 12 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:58.912828 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:58.919338  2030 raft_consensus.cc:2749] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:58.933194  2199 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:58.909315  1973 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:12:58.938171  2264 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:58.994040  2281 log.cc:927] Time spent T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:12:58.940183  2190 consensus_queue.cc:1048] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:12:58.978972 28902 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:58.980829  1795 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.961864  1851 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:12:59.001924 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 50533.2us
I20260709 13:12:58.932446  2292 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:58.986001 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 62766.2us
I20260709 13:12:59.002346  1851 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:59.010637 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 49122.2us
I20260709 13:12:59.010061 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:12:58.908386  2021 consensus_queue.cc:1059] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.098s
I20260709 13:12:59.005077  1795 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.030140 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:59.008778  1851 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:59.022174  2217 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:59.060170  1950 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.060853  1950 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.062819  1795 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.063158  1950 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.068212 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:59.069188 29007 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:12:59.070333  2174 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.068691  1839 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:59.074908  2174 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.075767  2198 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.076378  2198 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.076839 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:59.078524  2198 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.097317 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:59.079133  2198 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.068743  2053 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.080096  1950 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.083351  1821 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.116892 28902 catalog_manager.cc:5697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:59.082762  2029 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:12:59.097082  2153 consensus_queue.cc:1048] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:12:59.135106 29005 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
W20260709 13:12:59.147943  2296 log.cc:927] Time spent T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:12:59.149277  1151 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:59.108675  2053 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.118634  1950 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:59.120112  1821 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.115473  2198 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:12:59.210861 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 85181.3us
I20260709 13:12:59.229439  2319 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:59.202154  1950 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:59.321663  2174 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.329586 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.330498  1982 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:12:59.330551 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.331116  1982 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:59.331805 29150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:59.352193 29150 raft_consensus.cc:606] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:59.352624 29150 raft_consensus.cc:614] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:59.332365 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.357246  2202 raft_consensus.cc:3055] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:59.357668  2202 raft_consensus.cc:740] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:12:59.339056  1851 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:59.342890  2198 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:59.331588  2153 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.331961  1982 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.345439 28902 catalog_manager.cc:5697] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 12, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 12 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:12:59.350549  2295 log.cc:927] Time spent T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.290s	user 0.001s	sys 0.001s
I20260709 13:12:59.351094  1966 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:59.367771 29157 raft_consensus.cc:1240] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Rejecting Update request from peer ca45420e033847489ad989c1c6eed3ef for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:59.381484 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 225795us
W20260709 13:12:59.373816  2299 log.cc:927] Time spent T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.314s	user 0.004s	sys 0.000s
I20260709 13:12:59.393079 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:59.358501  2202 consensus_queue.cc:260] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:59.365729  2237 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:59.426029 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:59.431986 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:59.388334  2153 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.439065  2153 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:59.439807 29155 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:12:59.448698 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:59.416550  1982 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:59.460529 29076 raft_consensus.cc:2393] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:12:59.444685  2293 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:12:59.445037  2202 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:59.485024 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:12:59.444473  2195 consensus_queue.cc:1175] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "06ae4197fd784525b695a9b16c8094ea" responder_term: 11 status { last_received { term: 11 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 11 index: 12 } }
I20260709 13:12:59.444728  2217 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:12:59.488165 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 337723us
W20260709 13:12:59.495541 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 324180us
I20260709 13:12:59.497916 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:59.505616 28975 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:59.441753  2237 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:12:59.508239 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 54368us
I20260709 13:12:59.512564 28975 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:12:59.515476  2315 log.cc:927] Time spent T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:12:59.516947  2053 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.503860  2198 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:12:59.518111  2198 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:12:59.520224  2198 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:12:59.520817  2180 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:59.503952  2106 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.528316  2106 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.530328  2106 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:12:59.534804 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:12:59.538156 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 116976us
I20260709 13:12:59.520692  2198 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:59.545027 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:12:59.546232 29713 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:12:59.547793 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:12:59.551744  2237 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.554257 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.554448 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.562695 29153 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:12:59.554659 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.555045 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.555122 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.502839  2217 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.556588  2333 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.558774  1821 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.567189 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:59.569531 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:59.576932 29076 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:12:59.570327 28975 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:59.571961 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:59.574339 29152 raft_consensus.cc:2368] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 16. Current term is 17.
I20260709 13:12:59.577702 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.542296  2053 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:59.543428  2290 raft_consensus.cc:3055] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:59.582059 28975 leader_election.cc:400] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 16. Current term is 17.
I20260709 13:12:59.577867 29157 raft_consensus.cc:606] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:59.582666 29157 raft_consensus.cc:614] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:59.578691 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:12:59.577025 29003 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
I20260709 13:12:59.583642 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:59.584275 29156 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 17: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:12:59.588551 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:59.590176 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:12:59.595129 29007 raft_consensus.cc:1240] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:59.595363 28975 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:12:59.595314 29010 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:12:59.600735 28975 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:12:59.602602 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:59.574841  2198 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:59.605718 29005 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:12:59.603523 29081 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:12:59.577826  2254 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.636876  2158 consensus_queue.cc:1059] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:59.637370  2254 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.618127  2331 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:59.604239  2318 raft_consensus.cc:2749] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:59.620949 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:59.656028 29006 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:12:59.663996 29153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:59.664853 29153 raft_consensus.cc:606] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:12:59.665304 29153 raft_consensus.cc:614] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:59.639776  2254 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.603474  1821 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:59.656251  2331 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:59.589596  1950 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:59.585634  2290 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:12:59.674722  2290 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.675988  2290 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:59.677187  2331 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:59.678025  1950 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.678499  1950 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:59.606501  2053 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:59.582206  2333 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.636260  2217 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:59.647718 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.701437  2333 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:12:59.647588 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 70289.6us
I20260709 13:12:59.706461 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:59.706934 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:59.707244 29003 raft_consensus.cc:2393] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 17: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:12:59.708571 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:59.625385 29005 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:12:59.719195 29122 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
W20260709 13:12:59.806631 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 76103.4us
I20260709 13:12:59.705436 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:12:59.814185 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 108821us
W20260709 13:12:59.827945 29681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.828648 29153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "824fc688db264856b025a29c651eec78"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:12:59.829219 29153 raft_consensus.cc:606] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
W20260709 13:12:59.831537 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 164557us
I20260709 13:12:59.807927  2198 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:59.808239  2053 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:59.867274 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 263323us
I20260709 13:12:59.816135  2292 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:59.826912  2333 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:12:59.887960 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.870362  2198 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:59.881650  2292 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:59.868606  2217 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:12:59.890422  2333 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:12:59.901199  2254 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.922016  2254 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.886703  2053 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.926067  2053 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.917078  2217 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.928279 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:12:59.928653  2217 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.929008 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:12:59.931224  1950 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:12:59.935465 29009 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:12:59.935845 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
W20260709 13:12:59.937670 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45716.9us
I20260709 13:12:59.916432  2292 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:59.942708 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 70215.9us
I20260709 13:12:59.921060  2333 raft_consensus.cc:3060] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:59.990649  1950 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.014950 29122 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:00.016014  1839 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:00.016518  1839 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:00.017329  1839 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.022749  2331 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.033744 29076 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 17: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:13:00.034648  2331 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:12:59.984354  2254 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.037530 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:00.040573  1969 raft_consensus.cc:993] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:00.042622 29081 raft_consensus.cc:2393] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 13: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:13:00.043888 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 82195.4us
I20260709 13:13:00.029402  2239 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.060537  2239 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.062624  2239 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.034837  2333 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.048573  2290 raft_consensus.cc:1081] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:00.066258  2333 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.075165  2292 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.088382  2198 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.089938  1851 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.098196  2292 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.160481 29006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "824fc688db264856b025a29c651eec78"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:00.159089 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.162108 29002 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:00.168301 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.161823 29006 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:00.176427 29006 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:00.179296 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:00.180625 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 135764us
I20260709 13:13:00.182615 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:00.186275 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.187480 29004 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:00.188892 29002 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:13:00.207190 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.208957 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.210119 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.215713 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.216531 29153 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:13:00.217133 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.218168 29152 raft_consensus.cc:2368] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 17. Current term is 18.
I20260709 13:13:00.219627 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:00.233166 29158 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:13:00.241986 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:00.242467 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:00.242944 29716 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 11.
I20260709 13:13:00.245389 29051 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:00.252068  1821 raft_consensus.cc:2764] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:13:00.252465  1795 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.253371  1795 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.257493  1795 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.243675 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:00.265329 29714 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:00.258430  2021 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:00.271349  1966 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:00.269644 29005 raft_consensus.cc:2468] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
I20260709 13:13:00.274947  2269 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:00.278082  2264 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:00.274327  2254 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:00.280256 29004 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 18.
I20260709 13:13:00.270401 28975 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:00.284330 29006 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.286370  2021 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.279917  1966 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.284168  2254 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.301390 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.301826 29048 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:00.302546 29008 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:13:00.309051 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:00.311146  2346 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:00.312261  2323 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:00.313658  2323 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:00.298249 29717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:51170
I20260709 13:13:00.299104  2254 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:00.321591 29717 raft_consensus.cc:606] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:00.298904  1966 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:00.303999  2331 raft_consensus.cc:2804] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:00.306115  2021 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.334007 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.342203 29154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:00.342373  2292 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.342396 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:00.342933 29154 raft_consensus.cc:606] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:00.343432 29154 raft_consensus.cc:614] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:00.343272  2292 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:00.342229 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 75410.9us
I20260709 13:13:00.345470  2292 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.346719  2322 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:00.347724  2331 raft_consensus.cc:697] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:00.347591  2322 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.348956  2331 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.351217  2322 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.357420 29714 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:13:00.293380  2269 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.359946 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:00.364571 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:13:00.365020 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.366746 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:00.378943 29006 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.381145  2277 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.383363  2277 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.384588  2330 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.385298  2330 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.385761  2277 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.386977 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:00.387567  2330 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.387719 29159 raft_consensus.cc:3055] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:00.431216 29159 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:00.432494 29159 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.433511 29159 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:00.388319  2333 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.437080  2333 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.363071 29048 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:13:00.438946 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 82343.6us
I20260709 13:13:00.392297 29081 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:00.393716  2320 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.317283  2346 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.447742  2320 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.449347  2346 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.440448  2288 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:00.450352  2288 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.450807  2288 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:00.453481  2254 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.441972  2340 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.456207  2254 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.456754  2340 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.457985 29081 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:13:00.459201  2340 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.445618 28975 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:00.461421  2319 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:00.463285 28902 catalog_manager.cc:5697] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 15, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.397389 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:00.398285  1795 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.400022 29122 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:00.403723  2237 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:00.404570  2323 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.413738  2269 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.418769  1966 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.421383  2239 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.425470 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:00.430140  2217 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.358954  2264 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.444588 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.449949  2320 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.397250  2198 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.465814 28902 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 14, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 14 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:00.471256  1795 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:00.472431 28901 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52660 (request call id 249) took 41105 ms (client timeout 15000 ms). Trace:
I20260709 13:13:00.439996  2333 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:00.474376 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 113857us
I20260709 13:13:00.485713 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:00.475077  2316 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:00.475858 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 76005.9us
I20260709 13:13:00.475715 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.473068 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:00.477622 28976 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:00.473225  2237 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.491787 29159 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:13:00.497299  1123 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:00.504427 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.505033 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.495628  2237 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:00.507642 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.511637 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:00.503814  2239 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.510406 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.510927  1123 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.511623  1795 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.512328 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:00.524662 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.512161 28902 catalog_manager.cc:5697] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 12, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 12 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.524103 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:00.519136 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:00.519991 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:00.526504 29715 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:00.526527 29711 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:00.523398  2217 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.495177  2320 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.530225  2217 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.503647  2319 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:00.531096  2319 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:00.532150  2319 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.535526 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.536576 29003 raft_consensus.cc:2410] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:13:00.538156 29001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.538816 29001 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:13:00.539638 29007 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:00.542409  1151 raft_consensus.cc:2804] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:00.542968  1151 raft_consensus.cc:697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 18 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:00.544061  1151 consensus_queue.cc:237] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.545109  2239 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.530663  2320 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:00.557612 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.557801 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.558904 29008 raft_consensus.cc:2393] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 16: Already voted for candidate 06ae4197fd784525b695a9b16c8094ea in this term.
I20260709 13:13:00.556253 28902 catalog_manager.cc:5697] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 7 to 11, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.576906 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.579674 29715 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:13:00.580981 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.581701 29159 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:13:00.585773  2333 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.560642  2346 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:00.589216 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:00.594111 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.596791 29682 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:00.597498 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.598109 29155 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:13:00.599489  2202 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:00.599589 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:00.601779 29157 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 17: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:13:00.590600  2292 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.603233  1827 raft_consensus.cc:993] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: : Instructing follower befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:13:00.602994  2292 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.603718  1827 raft_consensus.cc:1081] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Signalling peer befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:13:00.605453  2292 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.600489 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.606626 29154 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 10.
I20260709 13:13:00.586526  2333 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.574590  2340 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.608250  1123 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.591212  2288 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.608428  2340 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.609057 29083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:39900
I20260709 13:13:00.609601 29083 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:00.609963 29083 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:00.610162  2288 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.610807  2340 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.613174 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.613448  2333 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.613804 29010 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:00.615523  2320 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.615590 28902 catalog_manager.cc:5697] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.616617  2358 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.617337  2358 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.563685  1795 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.619000 29048 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:00.619534  2358 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.619395  1795 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.620851 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.621639 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:00.621747 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.622279  2264 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.605371  1966 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.622306 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.628397 29076 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 16.
I20260709 13:13:00.630733 29711 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:13:00.618378 28902 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:00.632717 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 43573us
I20260709 13:13:00.636942 28976 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:00.644516 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.646625  1123 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:00.649590 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:00.651707 29158 raft_consensus.cc:2393] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 17: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:13:00.634238  2358 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:00.653708 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.655992 28902 catalog_manager.cc:5697] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.636065  1151 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:00.664705 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.665885 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:00.666662 29711 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:13:00.670203 29051 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:00.669066 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.683173 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:00.642546  2320 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.683835 29158 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:13:00.683936 29003 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:00.666119  1151 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.676192  2333 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.698482 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.698752 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.699514 29005 raft_consensus.cc:2410] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:13:00.683080  2198 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:00.700090 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:00.704140 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:13:00.705086 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 52259.6us
I20260709 13:13:00.689258  2217 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:00.710410  2030 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.711112  1950 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.711325  2030 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.711709  1950 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.712266  2333 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.713745  2030 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:13:00.485822 28901 rpcz_store.cc:269] 0709 13:12:19.367044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:19.367317 (+   273us) service_pool.cc:224] Handling call
0709 13:13:00.472132 (+41104815us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:55.104160 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:55.168309 (+ 64149us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:55.168348 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:12:55.168374 (+    26us) tablet.cc:662] Decoding operations
0709 13:12:55.168982 (+   608us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:55.169047 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:55.169078 (+    31us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:12:55.169447 (+   369us) tablet.cc:701] Row locks acquired
0709 13:12:55.169476 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:12:55.169634 (+   158us) write_op.cc:270] Start()
0709 13:12:55.169772 (+   138us) write_op.cc:276] Timestamp: P: 1783602775169581 usec, L: 0
0709 13:12:55.169795 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:12:55.170500 (+   705us) log.cc:844] Serialized 1308 byte log entry
0709 13:12:55.833924 (+663424us) op_driver.cc:464] REPLICATION: finished
0709 13:12:56.601003 (+767079us) write_op.cc:301] APPLY: starting
0709 13:12:56.601103 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:56.603373 (+  2270us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:56.603485 (+   112us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:56.828333 (+224848us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:56.828580 (+   247us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:12:56.831985 (+  3405us) write_op.cc:312] APPLY: finished
0709 13:12:56.900161 (+ 68176us) log.cc:844] Serialized 41 byte log entry
0709 13:12:57.118134 (+217973us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:57.130290 (+ 12156us) write_op.cc:454] Released schema lock
0709 13:12:57.131969 (+  1679us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:57.159996 (+ 28027us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:57.172414 (+ 12418us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1113234,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":711820,"apply.run_cpu_time_us":40376,"apply.run_wall_time_us":547368,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16667,"num_ops":3,"prepare.queue_time_us":295000,"prepare.run_cpu_time_us":44521,"prepare.run_wall_time_us":164027,"raft.queue_time_us":468226,"raft.run_cpu_time_us":46659,"raft.run_wall_time_us":660381,"replication_time_us":632626,"thread_start_us":1115069,"threads_started":5,"wal-append.queue_time_us":348791,"wal-append.run_cpu_time_us":34517,"wal-append.run_wall_time_us":2209926}]]}
I20260709 13:13:00.739249  2320 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.706513 28902 catalog_manager.cc:5697] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.710364 29050 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:00.745805 29158 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 12.
I20260709 13:13:00.701401  1151 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:00.750530 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 279985us
I20260709 13:13:00.748785 29083 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.757725 29009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.709662  2346 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:00.734127  2333 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.742039  2198 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:00.745223  2217 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:00.763527 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.760953 28902 catalog_manager.cc:5697] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.766240 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:00.766973 29711 raft_consensus.cc:2368] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12.
I20260709 13:13:00.752065  2323 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:00.772847 29716 raft_consensus.cc:1240] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:00.774215 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:00.761304  2346 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:00.776587 29083 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:13:00.778252 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 109979us
I20260709 13:13:00.782258 29715 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:13:00.782361 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:00.769356  2330 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:00.783001 29714 raft_consensus.cc:3055] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:00.783397 29714 raft_consensus.cc:740] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:13:00.783934  1795 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.784639 29714 consensus_queue.cc:260] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.786067 29714 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:00.789999  1982 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:00.777843  2346 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:00.793375  2323 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.807083 29009 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:13:00.791405  1982 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.798111 28902 catalog_manager.cc:5697] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.809641  1982 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.810019 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.813910  2323 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.808523 28902 catalog_manager.cc:5697] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 13, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.816763 29050 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:00.816785 29125 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:00.819666 29714 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:13:00.821202 29051 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:00.828430 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:00.824401  2320 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.839514  1851 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:00.832657  2333 raft_consensus.cc:2804] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:00.839972  1851 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.825910  2030 consensus_queue.cc:1059] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:13:00.838300  2330 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:00.845471  1851 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:00.848171 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 55238us
I20260709 13:13:00.848560  2358 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.850231  2292 raft_consensus.cc:3055] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:00.833460 28902 catalog_manager.cc:5697] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 7 to 12, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 12 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:00.832623  2237 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.863044  2292 raft_consensus.cc:740] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:13:00.878481  2360 log.cc:927] Time spent T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:13:00.865716  2358 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:00.880975 29050 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:00.882858 28902 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:00.875072  2333 raft_consensus.cc:697] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:00.888310  1795 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:00.842118  2217 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.843451  2198 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.879500  2292 consensus_queue.cc:260] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.900779  2319 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:00.903681  2319 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.905746  2292 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:00.906699  2319 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.910632  2346 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.902369 28902 catalog_manager.cc:5697] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 11, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:00.928012 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:00.932323 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:00.932979  1950 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.938488  2358 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.933110 29076 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 18: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:13:00.940851  2358 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:00.925365  2237 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:00.951747 29125 leader_election.cc:400] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12.
I20260709 13:13:00.952091 29125 leader_election.cc:403] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:00.952464 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:00.952756 29125 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:00.953011 29715 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:13:00.953877  2351 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) with higher term. Message: Invalid argument: T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 for earlier term 11. Current term is 12.
I20260709 13:13:00.954530  2351 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:00.954623 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:00.955179 29716 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:00.944639 29050 leader_election.cc:400] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 17. Current term is 18.
I20260709 13:13:00.948482 29005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:00.957202 29050 leader_election.cc:403] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:00.948500 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.960536 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.958593  1151 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:00.949628 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.977910 29010 raft_consensus.cc:3060] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:00.958527 29008 raft_consensus.cc:3060] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:00.961258 29006 raft_consensus.cc:3060] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:00.961625  2346 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:00.968655 29002 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:00.971100 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:00.974656 29078 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:00.989250  2237 raft_consensus.cc:3060] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:13:00.990031  2333 consensus_queue.cc:237] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:00.993727 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:00.994524 29083 raft_consensus.cc:2368] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for earlier term 16. Current term is 17.
I20260709 13:13:00.995658 29683 leader_election.cc:400] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 election: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for earlier term 16. Current term is 17.
I20260709 13:13:00.995949 29683 leader_election.cc:403] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:13:00.997326  2237 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 17. Current term is 18.
I20260709 13:13:00.977234 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:00.999296 29010 raft_consensus.cc:2468] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:13:01.004240  2363 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:01.005673  2363 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:01.006047  2363 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:01.008217 28902 catalog_manager.cc:5697] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 3 to 7, term changed from 7 to 8, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66), VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:01.010416 28902 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 12, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 12 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:01.013527  2363 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:01.016098  2363 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:01.017546 29716 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
I20260709 13:13:01.017776  2053 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:01.018103  2264 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939) with higher term. Message: Invalid argument: T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef for earlier term 16. Current term is 17.
I20260709 13:13:01.021919  2237 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:13:00.989360 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44927.4us
I20260709 13:13:01.026161  1151 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:01.027463 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:13:01.027834 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.028394  2320 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:01.029910 29006 raft_consensus.cc:2468] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
W20260709 13:13:01.040625 28895 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ca45420e033847489ad989c1c6eed3ef is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:01.042109 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:00.948743 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:01.033586 29051 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:01.030910 29050 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.032294 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:01.046391 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.043404 29006 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:13:01.047214 28902 catalog_manager.cc:5697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 3 to 15, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.058355 29050 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.065141 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.065742 29155 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:13:01.062443 28902 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 12, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 12 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:01.066901 29159 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:01.070304 28975 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:01.069379 29050 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.075572 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.075819 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:01.064997  1851 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:01.075269  2198 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:01.072844 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.080781  1851 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:01.079941  2198 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting leader election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:01.082491  2198 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:01.084872  2345 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:13:01.085461  2330 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:01.085495  2340 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:01.087249  2340 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.085402  1123 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:01.089243 29153 raft_consensus.cc:2368] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13.
I20260709 13:13:01.088538  1123 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:01.092805 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:01.093413 29003 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:01.059504  2320 raft_consensus.cc:2804] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:01.074838  2237 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:01.066715  1795 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:01.084688 29152 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:13:01.099273 29008 raft_consensus.cc:2468] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:01.101493 28975 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:01.103644 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:01.090831  2358 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:01.106247 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 69448.5us
I20260709 13:13:01.086470  2330 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.107609 29048 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:01.104558  2320 raft_consensus.cc:697] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:01.112005  2358 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:01.112692  2358 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.112818  2320 consensus_queue.cc:237] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:01.114775  2358 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:01.115446  2358 raft_consensus.cc:2804] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:01.085778  2288 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:01.094110  2053 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:13:01.120939 29078 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:01.124310  1123 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:01.109161  2333 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:01.118660  1950 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:01.128450  1950 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.128852  1950 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:01.129388  2340 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:01.131865 28975 leader_election.cc:400] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13.
I20260709 13:13:01.132264 28975 leader_election.cc:403] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:13:01.115919  2358 raft_consensus.cc:697] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:01.134148 29716 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 12.
I20260709 13:13:01.131770  2288 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.141515  2288 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:01.143111  2330 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:01.149773 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:01.132580 28902 catalog_manager.cc:5697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 15, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.156211 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:01.153702  2053 consensus_queue.cc:1059] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:13:01.160158 28902 catalog_manager.cc:5697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.167951 29122 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:01.182327  1123 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:01.205454 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 136133us
W20260709 13:13:01.206653 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 99140.4us
I20260709 13:13:01.208171 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:01.210836 29003 raft_consensus.cc:2468] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:13:01.212064 29122 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:01.212793  1123 raft_consensus.cc:2804] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:01.213265  1123 raft_consensus.cc:697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:01.216161 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:01.216328 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:01.216794 29078 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 18.
I20260709 13:13:01.217422 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:01.217682 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:01.219161 29152 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:01.214495 28902 catalog_manager.cc:5697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 11 to 15, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.219794  2358 consensus_queue.cc:237] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.213346  2363 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea for earlier term 12. Current term is 13.
W20260709 13:13:01.234942  2367 log.cc:927] Time spent T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.162s	user 0.003s	sys 0.000s
I20260709 13:13:01.238543  2198 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:01.239697  2366 log.cc:927] Time spent T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:13:01.246075  2198 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:01.248212 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:01.248996 29078 raft_consensus.cc:2393] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 06ae4197fd784525b695a9b16c8094ea in current term 18: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:13:01.261642 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:01.271299 28902 catalog_manager.cc:5697] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 8 to 12, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 12 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:01.274158 28902 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.278049  2371 raft_consensus.cc:493] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:01.279871  2370 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:01.294896  1950 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.296134 29007 raft_consensus.cc:2368] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:13:01.296233  2370 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.283222  1123 consensus_queue.cc:237] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:01.307983 29155 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:13:01.307451 28902 catalog_manager.cc:5697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.298432  2370 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:01.326264 29008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:01.326485 28902 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 10, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 10 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.349694  1950 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:01.324570  2371 raft_consensus.cc:515] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.361979  2371 leader_election.cc:290] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:01.362573 29010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:01.364429 29048 leader_election.cc:400] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:13:01.358712 28902 catalog_manager.cc:5697] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 1 to 13. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.364770 29048 leader_election.cc:403] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:01.364992  2319 raft_consensus.cc:3055] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:01.365340  2319 raft_consensus.cc:740] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:01.365605  2371 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:01.366609 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:01.368763 29711 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:01.372437 29077 raft_consensus.cc:1240] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:01.379685  1795 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:01.381675 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:01.382414 29157 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:01.407140 28975 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:01.365964 28902 catalog_manager.cc:5697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.425731 28902 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.409914  2198 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:13:01.442463 28902 catalog_manager.cc:5697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.429970  2375 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:13:01.451069  2376 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:13:01.454463 29008 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:01.453294 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:01.459798 29158 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:13:01.460305 29158 pending_rounds.cc:85] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Aborting all ops after (but not including) 14
I20260709 13:13:01.459322  2320 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:13:01.464881 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:01.451824  2375 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:01.466284  2375 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.468827 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:01.472601  2371 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941) with higher term. Message: Invalid argument: T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:13:01.473008 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:01.474371 29080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:01.474978 29080 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 18.
I20260709 13:13:01.475986 29081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:01.477196 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.460582 29158 pending_rounds.cc:107] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:13:01.485095 29158 raft_consensus.cc:2889] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:01.485699 28902 catalog_manager.cc:5697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 12, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 12 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.489934 28902 catalog_manager.cc:5697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.491321 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:01.492076 29079 raft_consensus.cc:2393] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 17: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:13:01.508078 29715 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:13:01.525034 28902 catalog_manager.cc:5697] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.521842  2322 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:13:01.537634  2319 consensus_queue.cc:260] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.539438  2319 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:01.548769 29711 raft_consensus.cc:1275] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:13:01.554159 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 44437.4us
I20260709 13:13:01.554800 29085 raft_consensus.cc:2468] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 18.
I20260709 13:13:01.558107 29084 raft_consensus.cc:1275] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:13:01.584184  2320 consensus_queue.cc:1048] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:01.593210 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:01.602643 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.603025 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.603207 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.603984 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.604983 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:01.605492 29158 raft_consensus.cc:3060] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:01.608539 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.610385 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.611160 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.611517 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.612360 29153 raft_consensus.cc:2410] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:13:01.614879 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.629136 29083 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:13:01.636679 29156 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:01.630959 29125 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:01.641517 29158 raft_consensus.cc:2468] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:13:01.644575 29157 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:13:01.651975 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:01.652789 29155 raft_consensus.cc:2393] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 16: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:13:01.656178 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:01.660384  1851 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:01.661418 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.684155 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.684675 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.685688 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa4501421f5146e4a7b79fd8697f8f57" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.686565 29050 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.687932  2322 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:01.688956 29154 raft_consensus.cc:1275] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:01.691841 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:01.692792  2345 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:01.693240  2345 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.693619  2345 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:01.698706 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:01.715682 28975 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:01.718422  2322 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:01.719072  2322 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:01.720150  2322 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.724406  2292 raft_consensus.cc:2749] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:01.728180 29125 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:01.729030  2351 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:01.730473  2351 consensus_queue.cc:1048] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:01.740485 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:01.741443 29157 raft_consensus.cc:2368] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
W20260709 13:13:01.742785 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 88420.8us
I20260709 13:13:01.744315 28902 catalog_manager.cc:5697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:01.756736 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 70263.2us
I20260709 13:13:01.758360 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.759562 28902 catalog_manager.cc:5697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.762851 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.765482  2370 raft_consensus.cc:2749] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:01.773289  2371 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:01.776106 29714 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:01.751479  2351 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:01.786496 29158 raft_consensus.cc:1240] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:13:01.787118 29002 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:13:01.790968 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.791821  2320 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:01.798334 29050 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.800515 28902 catalog_manager.cc:5697] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 4 to 12, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 12 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:01.802578 28902 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:01.805305 29050 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.809427 29050 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.812403 29050 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.812335 28902 catalog_manager.cc:5697] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.815405 29153 raft_consensus.cc:1275] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:13:01.819340 29050 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.820766 29714 raft_consensus.cc:3060] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:01.823266 29050 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:01.825886 29004 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:13:01.816768  2371 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:01.804580  2320 raft_consensus.cc:2749] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:01.805815  2331 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:01.814320  1795 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:01.795147  2330 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:13:01.805027  2217 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:13:01.820852  2369 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:01.794978  2351 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:01.832681  1123 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:13:01.850442 29050 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:01.854358  2358 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:01.936106  2386 log.cc:927] Time spent T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:13:01.936179  2358 raft_consensus.cc:2804] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:01.936829  2358 raft_consensus.cc:697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:01.937635  2358 consensus_queue.cc:237] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:01.930666  2351 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:01.841801  2370 raft_consensus.cc:2749] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:01.934317 29714 raft_consensus.cc:1275] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:13:01.946355  2345 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:01.958972 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 102081us
I20260709 13:13:01.963201 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:01.965649 29153 raft_consensus.cc:1240] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Rejecting Update request from peer 06ae4197fd784525b695a9b16c8094ea for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:01.975875  2345 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:01.987550 29076 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:13:02.061386 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 106521us
I20260709 13:13:02.064836  1151 consensus_queue.cc:1048] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:13:02.065414 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 62325.4us
W20260709 13:13:02.066495 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 216162us
I20260709 13:13:01.975925  2198 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:02.078059 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 75970.6us
W20260709 13:13:02.080737 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 75222.2us
W20260709 13:13:02.082882 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 93070.6us
I20260709 13:13:02.092726 29002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:02.096254 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:13:02.134179 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 60167.4us
I20260709 13:13:02.137866 29050 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:02.128731  1966 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:02.084100  2375 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:13:02.084339  2198 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:02.156440 29050 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:02.131034  2371 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:13:02.168849 29050 leader_election.cc:304] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:02.155709  1966 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:02.174460  2198 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:02.185716 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:02.188881 29155 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 17.
I20260709 13:13:02.190675 29050 leader_election.cc:400] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957) with higher term. Message: Invalid argument: T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b for earlier term 15. Current term is 16.
I20260709 13:13:02.151271  2397 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:02.184856  2320 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:02.212611 28975 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:02.180930  1795 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:02.198693  2395 raft_consensus.cc:2749] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:02.160456  2322 consensus_queue.cc:1059] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:13:02.229621  2403 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:02.230221  2403 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:02.234761  2202 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:02.235357  2202 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:02.236941  2394 log.cc:927] Time spent T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:02.239845  2404 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:02.240267  2404 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.240576  2404 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:02.240484 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:02.241210 29155 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:13:02.245333 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:02.202646  2331 consensus_queue.cc:1048] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:13:02.252729  2198 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:02.253111  2198 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:02.253847  2198 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:02.256330  2403 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:02.260125 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:02.260965 29716 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:13:02.202977  2396 consensus_queue.cc:1048] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:13:02.263053 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:02.270462  1151 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:02.270859  1151 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:02.271232  1151 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:02.272382 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 169619us
W20260709 13:13:02.278249 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 205662us
I20260709 13:13:02.255923  1966 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:02.296360 29004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:02.247396  2202 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:02.271000  2403 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:02.333976  1151 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:02.335913 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:02.339351 29085 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 17.
W20260709 13:13:02.340178 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 95436.2us
W20260709 13:13:02.335590 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 49273.4us
I20260709 13:13:02.350257  1151 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:02.342520 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:02.358012 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:02.369352 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:02.373950 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:02.322853  2403 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:02.392365 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:02.392985 29078 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:02.384613  2403 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:02.533569 28975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:02.550542  2407 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:13:02.553654  2404 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:02.509874  2396 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:13:02.618523 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 115661us
I20260709 13:13:02.633724 29680 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:02.634516 29078 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 19.
I20260709 13:13:02.580900  2396 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:02.697346  2419 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:13:02.700707  2198 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:02.713034  2396 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:02.649600  2410 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:02.721179  2403 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:02.722952 29715 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 15: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:13:02.729504  2403 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:02.633936  2404 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:02.773730 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 278120us
W20260709 13:13:02.778446 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 115194us
W20260709 13:13:02.782231 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 140521us
I20260709 13:13:02.783317 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:02.784915 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:02.789002  2428 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:02.791007 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:02.794968 29715 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:02.789561  2428 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:02.815698  2320 consensus_queue.cc:1059] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:02.817507  2320 raft_consensus.cc:3055] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:02.817876  2320 raft_consensus.cc:740] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:02.818683  2320 consensus_queue.cc:260] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:02.821676 29003 raft_consensus.cc:1275] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:02.819993  2320 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:02.824409  2428 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:02.825275 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:02.832768 29078 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:02.834628 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:02.835153 29078 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:02.835583 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:02.836874  2292 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:02.842224 29078 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:13:02.846237 29125 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:02.848435  1151 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:02.848914  1151 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:02.849293 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:02.849742  1151 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:02.852288  2417 consensus_queue.cc:1048] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:02.856442 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:02.861681 29010 pending_rounds.cc:85] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 8
I20260709 13:13:02.862016 29010 pending_rounds.cc:107] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:13:02.862267 29010 raft_consensus.cc:2889] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:02.865036 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 99704.8us
I20260709 13:13:02.867530  1123 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:02.870920 28976 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:02.874010  2375 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:02.881477  2375 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:02.872798  2198 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:13:02.895577 29158 raft_consensus.cc:1240] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting Update request from peer befda4a6c8344c26b0f139e4e1686b3b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:02.895995 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:02.899900 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:02.902354 29152 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ca45420e033847489ad989c1c6eed3ef in current term 15: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:13:02.884317  2375 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:02.981704 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 43703.8us
I20260709 13:13:03.002898 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:03.003551 29157 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:03.007899 29152 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:13:03.015642 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 145751us
I20260709 13:13:03.033640 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:03.055316 28975 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:03.060334 29050 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:03.086329 29682 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:03.082830  2322 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:03.102399 29157 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
W20260709 13:13:03.105168 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44931.1us
W20260709 13:13:03.109908 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 43958.3us
I20260709 13:13:03.097075  2423 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:03.109256  2397 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:03.132014  2436 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:13:03.184206 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 308624us
I20260709 13:13:03.132344  2411 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
W20260709 13:13:03.224496  2445 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:13:03.255391 29157 raft_consensus.cc:1275] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:13:03.265147  2444 log.cc:927] Time spent T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:13:03.266610  2397 consensus_queue.cc:1048] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 13:13:03.279789  2447 log.cc:927] Time spent T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 13:13:03.292182 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 116743us
W20260709 13:13:03.308533 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 131587us
W20260709 13:13:03.385007  2442 log.cc:927] Time spent T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.350s	user 0.003s	sys 0.004s
W20260709 13:13:03.387020  2439 log.cc:927] Time spent T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.352s	user 0.003s	sys 0.000s
I20260709 13:13:03.433856 29076 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:13:03.441211 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:03.623476 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 166449us
W20260709 13:13:03.630911 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 194679us
I20260709 13:13:03.657119 29157 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:13:03.664353  2322 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:03.689062  2322 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:03.714814 29002 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:13:03.715271 29002 pending_rounds.cc:85] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Aborting all ops after (but not including) 12
I20260709 13:13:03.715523 29002 pending_rounds.cc:107] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:13:03.715756 29002 raft_consensus.cc:2889] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:03.727284 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 79601.6us
I20260709 13:13:03.734536  1123 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:03.771893 29077 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:13:03.805927 29154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:03.806567 29154 raft_consensus.cc:606] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:03.806895 29154 raft_consensus.cc:614] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:03.811113 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ed915f0be134d71b36d8fc8f7b5830b"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:03.811666 29157 raft_consensus.cc:606] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:03.812031 29157 raft_consensus.cc:614] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:03.798599  2446 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:13:03.816004 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:03.816532 29157 raft_consensus.cc:606] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:03.816857 29157 raft_consensus.cc:614] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:03.823992 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce7f7e5792294b1f99a1bf4ee332a76c"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:03.832540 29157 raft_consensus.cc:606] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:03.843653 29157 raft_consensus.cc:614] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:03.866377 29154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:03.867105 29154 raft_consensus.cc:606] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:03.867452 29154 raft_consensus.cc:614] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:03.879709 29716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:51170
I20260709 13:13:03.880617 29716 raft_consensus.cc:606] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:03.881086 29716 raft_consensus.cc:614] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:03.882710 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:13:03.884508 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:04.399130 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 80906.3us
W20260709 13:13:04.410799 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 90249.2us
I20260709 13:13:04.694203  2456 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.725466  2456 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:04.806108  2456 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:04.942622  2478 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.943243  2478 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:04.949352  2478 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:04.969743 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:04.970521 29716 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:13:04.980860  2415 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:04.981506  2415 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:04.981724 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:04.985834  2415 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:04.975414 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:04.989945 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:04.998298 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:04.998862 29078 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:13:05.001431 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:05.010694 29006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "06ae4197fd784525b695a9b16c8094ea" is_pre_election: true
I20260709 13:13:04.991003  2461 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:05.019472  2461 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.030766  2461 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:05.062978 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:05.085564 29122 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:05.087191  2441 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:05.099903 29155 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:05.127900 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45588.6us
W20260709 13:13:05.148373 28902 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52676 (request call id 246) took 14074 ms (client timeout 15000 ms). Trace:
I20260709 13:13:05.150328 29004 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:05.151803  2443 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:05.156729 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 027a3f0877db4e259d94a8e296822644 with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:34914
W20260709 13:13:05.159816 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 89214.1us
W20260709 13:13:05.148837 28902 rpcz_store.cc:269] 0709 13:12:51.074473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.075488 (+  1015us) service_pool.cc:224] Handling call
0709 13:13:05.148273 (+14072785us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:02.291156 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:02.292113 (+   957us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:02.292179 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:13:02.292206 (+    27us) tablet.cc:662] Decoding operations
0709 13:13:02.666752 (+374546us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:02.666870 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:02.666911 (+    41us) tablet.cc:685] Acquiring locks for 41 operations
0709 13:13:02.670933 (+  4022us) tablet.cc:701] Row locks acquired
0709 13:13:02.670991 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:13:02.671211 (+   220us) write_op.cc:270] Start()
0709 13:13:02.671396 (+   185us) write_op.cc:276] Timestamp: P: 1783602782671133 usec, L: 0
0709 13:13:02.671429 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:13:02.756185 (+ 84756us) log.cc:844] Serialized 15928 byte log entry
0709 13:13:03.343939 (+587754us) op_driver.cc:464] REPLICATION: finished
0709 13:13:03.438854 (+ 94915us) write_op.cc:301] APPLY: starting
0709 13:13:03.438985 (+   131us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:03.907398 (+468413us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:03.907527 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:04.142138 (+234611us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:04.142731 (+   593us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=82,key_file_lookups=82,delta_file_lookups=0,mrs_lookups=0
0709 13:13:04.142801 (+    70us) write_op.cc:312] APPLY: finished
0709 13:13:04.143547 (+   746us) log.cc:844] Serialized 348 byte log entry
0709 13:13:04.144505 (+   958us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:04.144859 (+   354us) write_op.cc:454] Released schema lock
0709 13:13:04.144949 (+    90us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:04.145453 (+   504us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:04.145492 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1638739,"reported_tablets":51,"spinlock_wait_cycles":196480,"child_traces":[["op",{"apply.queue_time_us":89349,"apply.run_cpu_time_us":56878,"apply.run_wall_time_us":706588,"cfile_cache_hit":166,"cfile_cache_hit_bytes":175507,"num_ops":41,"prepare.queue_time_us":70204,"prepare.run_cpu_time_us":31971,"prepare.run_wall_time_us":625818,"raft.queue_time_us":117166,"raft.run_cpu_time_us":12972,"raft.run_wall_time_us":124056,"replication_time_us":668816,"thread_start_us":360874,"threads_started":4,"wal-append.queue_time_us":228158,"wal-append.run_cpu_time_us":16870,"wal-append.run_wall_time_us":2148631}]]}
I20260709 13:13:05.178591 28895 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 027a3f0877db4e259d94a8e296822644 with cas_config_opid_index 3: aborting the task: latest config opid_index 7; task opid_index 3
I20260709 13:13:05.198580 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:05.199270 29711 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:13:05.204089 29715 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:05.183928  2461 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:05.206130 29154 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 16: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:13:05.212185  2461 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:05.217576 29051 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:05.218426  2496 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:05.218860  2496 raft_consensus.cc:493] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.240561  2496 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:05.255537 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:05.256140 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:05.261061  2443 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:05.256279 29717 raft_consensus.cc:3060] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:05.266077  2497 tablet_replica.cc:333] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:05.267326  2497 raft_consensus.cc:2243] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.268231  2497 raft_consensus.cc:2272] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.271697 29717 raft_consensus.cc:2468] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:13:05.275205  2496 raft_consensus.cc:515] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:05.278088 29077 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 16: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:13:05.278715  2446 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index 3 to 9, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.281909 29125 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:05.281759 29004 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 3 to 9, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.283128  2461 raft_consensus.cc:2804] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:05.283620  2461 raft_consensus.cc:697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:05.284451  2461 consensus_queue.cc:237] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:05.289664 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:05.290406  2496 leader_election.cc:290] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:05.291380 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4c87bb2e4a4f2ca684feff1fae17cf" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:05.292932 29715 raft_consensus.cc:2393] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 16: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:13:05.293975 29051 leader_election.cc:304] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:05.294883  2496 raft_consensus.cc:2749] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:05.305404 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab75774d33164f0b9c0269be22426768 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
I20260709 13:13:05.298424  2497 ts_tablet_manager.cc:1916] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:13:05.312829 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6c20 after lost signal to thread 27904
W20260709 13:13:05.314059 29738 debug-util.cc:398] Leaking SignalData structure 0x7b080031ca80 after lost signal to thread 28942
W20260709 13:13:05.315348 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08001e0800 after lost signal to thread 29034
W20260709 13:13:05.316560 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002f6160 after lost signal to thread 29109
W20260709 13:13:05.316864 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4da0 after lost signal to thread 29110
W20260709 13:13:05.317984 29738 debug-util.cc:398] Leaking SignalData structure 0x7b080033f780 after lost signal to thread 29183
W20260709 13:13:05.319253 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08003235e0 after lost signal to thread 29741
W20260709 13:13:05.319545 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08002d0200 after lost signal to thread 29742
W20260709 13:13:05.319993 29738 debug-util.cc:398] Leaking SignalData structure 0x7b08003c46e0 after lost signal to thread 2330
W20260709 13:13:05.320600 29738 debug-util.cc:398] Leaking SignalData structure 0x7b0800386c80 after lost signal to thread 2449
I20260709 13:13:05.359311 29717 raft_consensus.cc:2955] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 3 to 9, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.371203  2497 ts_tablet_manager.cc:1929] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.6
I20260709 13:13:05.371856  2497 log.cc:1199] T 027a3f0877db4e259d94a8e296822644 P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/027a3f0877db4e259d94a8e296822644
I20260709 13:13:05.404289 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 027a3f0877db4e259d94a8e296822644 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:05.406389 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 027a3f0877db4e259d94a8e296822644 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:34914
W20260709 13:13:05.408506 28903 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52660 (request call id 248) took 71635 ms (client timeout 15000 ms). Trace:
W20260709 13:13:05.409150 28903 rpcz_store.cc:269] 0709 13:11:53.772432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.773108 (+   676us) service_pool.cc:224] Handling call
0709 13:13:05.408082 (+71634974us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1422265,"reported_tablets":50}
I20260709 13:13:05.415174 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 027a3f0877db4e259d94a8e296822644 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
W20260709 13:13:05.438845 28908 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51350 (request call id 148) took 98895 ms (client timeout 15000 ms). Trace:
I20260709 13:13:05.446856 28902 catalog_manager.cc:5697] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 8 to 13, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 13 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:13:05.448089 28908 rpcz_store.cc:269] 0709 13:11:26.543654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:26.543873 (+   219us) service_pool.cc:224] Handling call
0709 13:13:05.438657 (+98894784us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63724,"reported_tablets":28}
I20260709 13:13:05.448957 28902 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 14 to 18, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 18 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.451439 28902 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 17, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.457921 28902 catalog_manager.cc:5697] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.459806 28902 catalog_manager.cc:5697] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 8 to 16, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 16 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:05.464139 28902 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 8 to 13, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 13 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.465399 28902 catalog_manager.cc:5697] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 15, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 15 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.466609 28902 catalog_manager.cc:5697] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.481302 28902 catalog_manager.cc:5697] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 12 to 13, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 13 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.482620 28902 catalog_manager.cc:5697] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.483952 28902 catalog_manager.cc:5697] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 8 to 14, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 14 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.485169 28902 catalog_manager.cc:5697] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 12 to 14, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 14 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.486685 28902 catalog_manager.cc:5697] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.520409 28902 catalog_manager.cc:5697] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.521854 28902 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 14 to 16, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 16 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.523078 28902 catalog_manager.cc:5697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 16, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 16 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.524288 28902 catalog_manager.cc:5697] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 14 to 16, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 16 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.525488 28902 catalog_manager.cc:5697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.526779 28902 catalog_manager.cc:5697] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 13 to 19, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 19 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.527987 28902 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 12, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 12 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.577860 28902 catalog_manager.cc:5697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.579661 28902 catalog_manager.cc:5697] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.599489 28902 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 13 to 14, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 14 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.613777 28902 catalog_manager.cc:5697] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 16, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 16 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.615010 28902 catalog_manager.cc:5697] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 5 to 10, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 10 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.616263 28902 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.617537 28902 catalog_manager.cc:5697] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 14 to 18, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 18 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.618706 28902 catalog_manager.cc:5697] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 11 to 13, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 13 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:05.619920 28902 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 10 to 15, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 15 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:05.621189 28902 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 3 to 9, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 12 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:05.745340 29717 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:05.747062  2461 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:05.835624 29085 raft_consensus.cc:1275] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:05.837309  2446 consensus_queue.cc:1048] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:06.204622 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet ab75774d33164f0b9c0269be22426768 with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:44418
I20260709 13:13:06.207793  2522 tablet_replica.cc:333] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:06.209193  2522 raft_consensus.cc:2243] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:06.211570  2522 raft_consensus.cc:2272] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:06.215296  2522 ts_tablet_manager.cc:1916] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:13:06.247933  2522 ts_tablet_manager.cc:1929] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.8
I20260709 13:13:06.248438  2522 log.cc:1199] T ab75774d33164f0b9c0269be22426768 P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/ab75774d33164f0b9c0269be22426768
I20260709 13:13:06.250267 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet ab75774d33164f0b9c0269be22426768 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:06.304201 28899 catalog_manager.cc:5697] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 9 to 13, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 13 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:06.320377 28899 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 9 to 10, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 10 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.330000 28899 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.338809 28899 catalog_manager.cc:5697] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.347160 28899 catalog_manager.cc:5697] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.372663 28899 catalog_manager.cc:5697] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.374691 28899 catalog_manager.cc:5697] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.379490 28899 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 11 to 15, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 15 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:06.408491 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet ab75774d33164f0b9c0269be22426768 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:44418
W20260709 13:13:06.411307 28907 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52674 (request call id 260) took 73500 ms (client timeout 15000 ms). Trace:
I20260709 13:13:06.413141 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet ab75774d33164f0b9c0269be22426768 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
W20260709 13:13:06.420506 28907 rpcz_store.cc:269] 0709 13:11:52.915927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.943527 (+ 27600us) service_pool.cc:224] Handling call
0709 13:11:54.893465 (+1949938us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54000f7dd4. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157ab090 00007f4115849f3a 00007f411b6001cc 00007f411b6fe37e 00007f4119bbf2a4 00007f4119bbf232 00007f4119bbf161 00007f4119bbf0dc 00007f4119bbe392 00007f4113b7c48c 00007f4113b7b8c6
0709 13:13:06.411098 (+71517633us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44953,"reported_tablets":51,"spinlock_wait_cycles":785867008}
I20260709 13:13:06.430089 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:06.538559 28899 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51350 (request call id 152) took 13947 ms (client timeout 15000 ms). Trace:
W20260709 13:13:06.539057 28899 rpcz_store.cc:269] 0709 13:12:52.594915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.677493 (+ 82578us) service_pool.cc:224] Handling call
0709 13:13:06.538490 (+13860997us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:06.431195 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:06.431765 (+   570us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:06.431803 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:13:06.431823 (+    20us) tablet.cc:662] Decoding operations
0709 13:13:06.432690 (+   867us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:06.432785 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:06.432815 (+    30us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:13:06.433443 (+   628us) tablet.cc:701] Row locks acquired
0709 13:13:06.433461 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:13:06.433606 (+   145us) write_op.cc:270] Start()
0709 13:13:06.433716 (+   110us) write_op.cc:276] Timestamp: P: 1783602786433558 usec, L: 0
0709 13:13:06.433737 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:13:06.434398 (+   661us) log.cc:844] Serialized 3194 byte log entry
0709 13:13:06.461375 (+ 26977us) op_driver.cc:464] REPLICATION: finished
0709 13:13:06.461956 (+   581us) write_op.cc:301] APPLY: starting
0709 13:13:06.462060 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:06.468086 (+  6026us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:06.468171 (+    85us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:06.488804 (+ 20633us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:06.489000 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:13:06.489040 (+    40us) write_op.cc:312] APPLY: finished
0709 13:13:06.489301 (+   261us) log.cc:844] Serialized 81 byte log entry
0709 13:13:06.489449 (+   148us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:06.489558 (+   109us) write_op.cc:454] Released schema lock
0709 13:13:06.489605 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:06.495760 (+  6155us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:06.495796 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":515726,"reported_tablets":28,"spinlock_wait_cycles":3840,"child_traces":[["op",{"apply.queue_time_us":321,"apply.run_cpu_time_us":10350,"apply.run_wall_time_us":33753,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33440,"num_ops":8,"prepare.queue_time_us":14920,"prepare.run_cpu_time_us":3486,"prepare.run_wall_time_us":3486,"replication_time_us":27459,"thread_start_us":27928,"threads_started":1}]]}
I20260709 13:13:06.599787 28901 catalog_manager.cc:5697] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 10 to 11, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 11 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:06.608286 28901 catalog_manager.cc:5697] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 4 to 6, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 8 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:06.611075 28901 catalog_manager.cc:5697] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 14 to 15, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 15 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:06.613212 28901 catalog_manager.cc:5697] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:06.644027 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:06.644696 29157 raft_consensus.cc:606] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:06.649124 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:06.649772 29157 raft_consensus.cc:606] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:06.670346 29154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:06.671180 29154 raft_consensus.cc:606] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:06.685567 29155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:06.686187 29155 raft_consensus.cc:606] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:06.700315 29717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931"
dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:51170
I20260709 13:13:06.701689 29717 raft_consensus.cc:606] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:06.707674 29155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:06.708264 29155 raft_consensus.cc:606] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:06.713201 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:13:06.715009 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:06.759081  2489 raft_consensus.cc:993] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:06.759608  2489 raft_consensus.cc:1081] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:06.761386 29004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:33750
I20260709 13:13:06.761926 29004 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:06.762249 29004 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:06.788695 29004 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:06.794390 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:06.794993 29717 raft_consensus.cc:3055] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:06.795281 29717 raft_consensus.cc:740] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:13:06.795979 29717 consensus_queue.cc:260] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:06.796288 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:06.796880 29155 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:06.798203 29717 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:06.799840  2517 raft_consensus.cc:993] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:06.800323  2516 raft_consensus.cc:1081] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:06.804507 29006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:06.805027 29006 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:06.805351 29006 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:06.811703 29155 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:13:06.829969 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet 2185927c34434da38aa9e7776e483b1b with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57656
I20260709 13:13:06.813390 29006 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:06.830583 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 027a3f0877db4e259d94a8e296822644 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:34914
I20260709 13:13:06.848851 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:06.822016 29717 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:13:06.849601 29077 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:06.822446 29004 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:06.834600 28901 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52660 (request call id 251) took 1912 ms. Trace:
I20260709 13:13:06.846132 28975 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:06.852078  2476 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:06.853976 29006 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:06.854074 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:06.854987 29155 raft_consensus.cc:3055] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:06.855345 29155 raft_consensus.cc:740] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:06.856146 29155 consensus_queue.cc:260] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:06.858003 29155 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:06.861929  2517 raft_consensus.cc:993] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:06.862534  2521 raft_consensus.cc:1081] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:06.862835 29077 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:13:06.865801 29004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:06.869179 29004 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:06.869647 29004 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:06.870640 28976 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:06.856228 28901 rpcz_store.cc:276] 0709 13:13:04.922457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.922777 (+   320us) service_pool.cc:224] Handling call
0709 13:13:06.834489 (+1911712us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:06.677700 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:06.678202 (+   502us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:06.678238 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:13:06.678257 (+    19us) tablet.cc:662] Decoding operations
0709 13:13:06.679527 (+  1270us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:06.679587 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:06.679614 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:13:06.680033 (+   419us) tablet.cc:701] Row locks acquired
0709 13:13:06.680071 (+    38us) write_op.cc:260] PREPARE: finished
0709 13:13:06.680224 (+   153us) write_op.cc:270] Start()
0709 13:13:06.680349 (+   125us) write_op.cc:276] Timestamp: P: 1783602786680178 usec, L: 0
0709 13:13:06.680374 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:13:06.681038 (+   664us) log.cc:844] Serialized 1674 byte log entry
0709 13:13:06.685693 (+  4655us) op_driver.cc:464] REPLICATION: finished
0709 13:13:06.686158 (+   465us) write_op.cc:301] APPLY: starting
0709 13:13:06.686242 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:06.710122 (+ 23880us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:06.710226 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:06.719128 (+  8902us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:06.719320 (+   192us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:13:06.719369 (+    49us) write_op.cc:312] APPLY: finished
0709 13:13:06.719605 (+   236us) log.cc:844] Serialized 49 byte log entry
0709 13:13:06.730268 (+ 10663us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:06.730377 (+   109us) write_op.cc:454] Released schema lock
0709 13:13:06.730421 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:06.735904 (+  5483us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:06.735946 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":83791,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":248,"apply.run_cpu_time_us":7069,"apply.run_wall_time_us":49689,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16720,"num_ops":4,"prepare.queue_time_us":9279,"prepare.run_cpu_time_us":3015,"prepare.run_wall_time_us":3625,"replication_time_us":3523}]]}
I20260709 13:13:06.879522  2517 raft_consensus.cc:993] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:06.880018  2517 raft_consensus.cc:1081] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:06.881701  2476 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:06.882659  2476 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:06.888285 29006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:06.888890 29006 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:06.889307 29006 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:06.891647 29155 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 15.
I20260709 13:13:06.898653 29004 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:06.904614 29004 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:06.905869 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:06.906471 29155 raft_consensus.cc:3055] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:06.906744 29155 raft_consensus.cc:740] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:06.907523 29155 consensus_queue.cc:260] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:06.913895 29155 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:06.921365 29155 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:13:06.924762 28975 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:06.924328  2529 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:06.927325  2529 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
W20260709 13:13:06.928483 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 57928.1us
I20260709 13:13:06.933147 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:06.933909 29077 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:06.938074 29006 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:06.946907 29077 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:13:06.949012 29006 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:06.950613 28906 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 2185927c34434da38aa9e7776e483b1b with cas_config_opid_index 4: aborting the task: latest config opid_index 6; task opid_index 4
W20260709 13:13:06.955729 28906 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51350 (request call id 151) took 35209 ms (client timeout 15000 ms). Trace:
I20260709 13:13:06.950186 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:06.956545  2529 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:06.943791  2476 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:06.968379  2526 tablet_replica.cc:333] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:06.969576  2526 raft_consensus.cc:2243] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:06.970427  2526 raft_consensus.cc:2272] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:06.960520 29155 raft_consensus.cc:3055] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:06.983927 29155 raft_consensus.cc:740] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:06.984668 29155 consensus_queue.cc:260] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:06.985805 29155 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:06.960094 28906 rpcz_store.cc:269] 0709 13:12:31.817685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.817974 (+   289us) service_pool.cc:224] Handling call
0709 13:13:06.955627 (+35137653us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17750,"reported_tablets":28}
I20260709 13:13:06.990388 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:06.990360 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 027a3f0877db4e259d94a8e296822644 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:06.994300 29085 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.000365 29155 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:13:07.004890 28975 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:06.988555  2476 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:07.021590  2526 ts_tablet_manager.cc:1916] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:13:07.023806 29085 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:13:07.056742  2532 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:07.057313  2532 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:07.058115  2532 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:07.105001 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 112673us
W20260709 13:13:07.111685 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 118461us
I20260709 13:13:07.116297  2476 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:07.162442  2531 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.190s	user 0.001s	sys 0.006s
I20260709 13:13:07.182274  2537 raft_consensus.cc:993] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
W20260709 13:13:07.205263 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 95001.1us
W20260709 13:13:07.219210 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 117049us
W20260709 13:13:07.225258 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 61815.8us
I20260709 13:13:07.218083  2538 raft_consensus.cc:1081] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:07.247303 29006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:07.248102 29006 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:07.248555 29006 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:07.292618 29006 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:07.298267 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 70470.9us
W20260709 13:13:07.308033 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 84101.2us
I20260709 13:13:07.310127 29006 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:07.321964  2526 ts_tablet_manager.cc:1929] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 8.5
I20260709 13:13:07.322522  2526 log.cc:1199] T 2185927c34434da38aa9e7776e483b1b P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/2185927c34434da38aa9e7776e483b1b
W20260709 13:13:07.342581 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 85630.8us
I20260709 13:13:07.357720 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:07.358309 29079 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:07.358688 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:07.359315 29155 raft_consensus.cc:3055] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:07.359608 29155 raft_consensus.cc:740] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
W20260709 13:13:07.360106 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 100886us
I20260709 13:13:07.360606 29155 consensus_queue.cc:260] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:07.361505 29155 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:07.370288 29151 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:07.389297 29079 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:13:07.390648 28976 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:07.395896 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet 2185927c34434da38aa9e7776e483b1b (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:07.393313 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet ab75774d33164f0b9c0269be22426768 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:44418
I20260709 13:13:07.443349 29155 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 16.
I20260709 13:13:07.404526  2530 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:07.449708 28904 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52674 (request call id 263) took 6886 ms. Trace:
I20260709 13:13:07.450122 28904 rpcz_store.cc:276] 0709 13:13:00.513419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.514017 (+   598us) service_pool.cc:224] Handling call
0709 13:13:00.813515 (+299498us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b54000f2490. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411271d640 00007f411b5f4b76 00007f411b6fec3e 00007f4119bbf2a4 00007f4119bbf232 00007f4119bbf161 00007f4119bbf0dc 00007f4119bbe392 00007f4113b7c48c 00007f4113b7b8c6
0709 13:13:07.399152 (+6585637us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59669,"reported_tablets":51,"spinlock_wait_cycles":4051200}
W20260709 13:13:07.492878 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 45203.4us
I20260709 13:13:07.498726 29717 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:07.406366  2476 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:13:07.495996  2530 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:07.512549  2530 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:07.529479  2528 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
W20260709 13:13:07.572800 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51799.2us
W20260709 13:13:07.574093 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 76966.6us
I20260709 13:13:07.580230 29151 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:07.587755 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet ab75774d33164f0b9c0269be22426768 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
W20260709 13:13:07.596531 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 77345.7us
W20260709 13:13:07.645234 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46956us
W20260709 13:13:07.668740 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51620.7us
I20260709 13:13:07.599089  2532 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
W20260709 13:13:07.692480 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 169392us
I20260709 13:13:07.704509 29085 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:13:07.711911 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 448720us
W20260709 13:13:07.712073 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 66921.1us
I20260709 13:13:07.720571 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:07.871073  2553 raft_consensus.cc:993] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:13:07.871587  2553 raft_consensus.cc:1081] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Signalling peer befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:13:07.909698 29085 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:44428
I20260709 13:13:07.910275 29085 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:07.910598 29085 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:07.922014 29085 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:07.935936 29157 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:13:07.930583  2572 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.254s	user 0.003s	sys 0.005s
I20260709 13:13:07.923965 28906 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 13 to 14, leader changed from ca45420e033847489ad989c1c6eed3ef (127.27.63.68) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 14 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:07.937011 29153 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:07.949326 29077 raft_consensus.cc:1240] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:07.972340 29085 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:07.983105  2551 consensus_queue.cc:1059] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:13:07.987901  2529 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:07.974979  2574 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:13:08.014504 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:08.027287 29155 raft_consensus.cc:3055] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:08.027673 29155 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:08.029281 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:08.029857 29003 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:08.034272 29155 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:08.035974 29155 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:08.047685 29155 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:08.051656 29050 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:08.060431 29077 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:08.061193 29085 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:13:08.056281  2551 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:13:08.069636 29003 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:08.057618  2525 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:08.092324 29152 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:08.108292  2528 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:08.105438  2525 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:08.110287  2540 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:08.110390  2525 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:08.132459  2528 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:13:08.118377  2540 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:13:08.193248 29077 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:08.205133  2532 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:13:08.238762 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet ab75774d33164f0b9c0269be22426768 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:44418
W20260709 13:13:08.286952 28905 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52674 (request call id 262) took 26174 ms (client timeout 15000 ms). Trace:
W20260709 13:13:08.287599 28905 rpcz_store.cc:269] 0709 13:12:42.088670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.088908 (+   238us) service_pool.cc:224] Handling call
0709 13:13:08.260268 (+26171360us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":588469,"reported_tablets":51}
W20260709 13:13:08.400626  2586 log.cc:927] Time spent T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 13:13:08.402375  2594 log.cc:927] Time spent T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:08.521641 28901 catalog_manager.cc:5697] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 14 to 15, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 15 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:08.523741 28901 catalog_manager.cc:5697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 15 to 16, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 16 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:08.560997 28901 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52660 (request call id 253) took 1041 ms. Trace:
I20260709 13:13:08.570807 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet ab75774d33164f0b9c0269be22426768 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:08.565519 28901 rpcz_store.cc:276] 0709 13:13:07.519600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.519754 (+   154us) service_pool.cc:224] Handling call
0709 13:13:08.560934 (+1041180us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:08.538244 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:08.538832 (+   588us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:08.538888 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:13:08.539062 (+   174us) tablet.cc:662] Decoding operations
0709 13:13:08.539502 (+   440us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:08.539562 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:08.539590 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:13:08.539909 (+   319us) tablet.cc:701] Row locks acquired
0709 13:13:08.539927 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:13:08.540073 (+   146us) write_op.cc:270] Start()
0709 13:13:08.540180 (+   107us) write_op.cc:276] Timestamp: P: 1783602788540014 usec, L: 0
0709 13:13:08.540204 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:13:08.540821 (+   617us) log.cc:844] Serialized 923 byte log entry
0709 13:13:08.542474 (+  1653us) op_driver.cc:464] REPLICATION: finished
0709 13:13:08.542949 (+   475us) write_op.cc:301] APPLY: starting
0709 13:13:08.543047 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:08.544397 (+  1350us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:08.544520 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:08.546096 (+  1576us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:08.546331 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:13:08.546401 (+    70us) write_op.cc:312] APPLY: finished
0709 13:13:08.546664 (+   263us) log.cc:844] Serialized 33 byte log entry
0709 13:13:08.546868 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:08.546989 (+   121us) write_op.cc:454] Released schema lock
0709 13:13:08.547049 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:08.547519 (+   470us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:08.547561 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":104489,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":265,"apply.run_cpu_time_us":4502,"apply.run_wall_time_us":4501,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8360,"num_ops":2,"prepare.queue_time_us":11407,"prepare.run_cpu_time_us":2733,"prepare.run_wall_time_us":2861,"replication_time_us":2138}]]}
I20260709 13:13:08.599732 29157 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:08.649580 28905 catalog_manager.cc:5697] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 12 to 13, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 13 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:08.654327 28905 catalog_manager.cc:5697] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 16, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 16 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } }
I20260709 13:13:08.665763 29007 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:13:08.667799  2607 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:08.687455 29084 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:08.696868  2537 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
W20260709 13:13:08.715328 29125 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a08e186ea0749ae979329103591a933. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:08.742377 29010 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:08.744879  2551 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:08.750355 29157 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:13:08.751672  2607 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:08.781582  2544 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:08.783223 29084 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:08.787428 28904 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 15 to 16, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 16 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:08.798838 29006 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:08.831465 28901 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 13, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 17 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
W20260709 13:13:08.906467 28900 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52660 (request call id 250) took 19131 ms (client timeout 15000 ms). Trace:
W20260709 13:13:08.907075 28900 rpcz_store.cc:269] 0709 13:12:49.775854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.778727 (+  2873us) service_pool.cc:224] Handling call
0709 13:13:08.906362 (+19127635us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27012,"reported_tablets":50,"spinlock_wait_cycles":23296}
I20260709 13:13:08.910952 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet 027a3f0877db4e259d94a8e296822644 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:34914
I20260709 13:13:08.892477 29084 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:08.922449 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet 027a3f0877db4e259d94a8e296822644 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
W20260709 13:13:08.944309 29051 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1a1cae5d4eb46b288c03a3675d24209. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:08.946482 29152 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:08.947976 29010 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:08.949698  2493 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:08.952831  2607 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:08.953351 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:08.983009  2605 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:09.016796 29157 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:09.041590 29156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a08e186ea0749ae979329103591a933"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:09.042373 29156 raft_consensus.cc:606] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:09.064327 29156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:09.064944 29156 raft_consensus.cc:606] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:09.086542 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:09.087304 29157 raft_consensus.cc:606] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:09.101184 28900 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index -1 to 11, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:09.111523 29157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:09.117512 29157 raft_consensus.cc:606] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 LEADER]: Received request to transfer leadership to TS befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:09.129750 29156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab75774d33164f0b9c0269be22426768"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:09.130332 29156 raft_consensus.cc:606] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:09.134424 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:09.133608 29010 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:09.136140 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:09.157415  2551 raft_consensus.cc:993] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:09.158094  2578 raft_consensus.cc:1081] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:09.159327  2578 raft_consensus.cc:993] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:13:09.159767  2551 raft_consensus.cc:1081] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 LEADER]: Signalling peer befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:13:09.161469 29084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:44428
I20260709 13:13:09.161998 29084 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:09.162352 29084 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:09.168772 29007 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab75774d33164f0b9c0269be22426768"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:09.169406 29007 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:09.169798 29007 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:09.182859 29084 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:09.185081 29156 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:09.199857 29003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:09.200551 29003 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:13:09.216531 29125 consensus_peers.cc:597] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa4501421f5146e4a7b79fd8697f8f57. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:09.216684 29006 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:09.218664 29078 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:09.218547  2551 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:09.220531  2519 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:09.228940 29084 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 20 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:09.231386 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:09.231956 29152 raft_consensus.cc:3055] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:09.232256 29152 raft_consensus.cc:740] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:09.232975 29152 consensus_queue.cc:260] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:09.234113 29152 raft_consensus.cc:3060] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:09.242307  2519 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:09.249658 29079 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:09.263916 29003 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 20.
I20260709 13:13:09.265254 29048 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:09.265906 29152 raft_consensus.cc:2468] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 20.
I20260709 13:13:09.266113  2605 raft_consensus.cc:2804] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:09.266615  2605 raft_consensus.cc:697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 20 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:09.298307  2577 raft_consensus.cc:993] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:13:09.299006  2577 raft_consensus.cc:1081] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Signalling peer befda4a6c8344c26b0f139e4e1686b3b to start an election
I20260709 13:13:09.301281 29084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c"
dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
 from {username='slave'} at 127.0.0.1:44428
I20260709 13:13:09.301792 29084 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:09.302136 29084 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:09.308962 28901 catalog_manager.cc:5697] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 9, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:09.292356 29007 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:09.317081 29007 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Requested vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:09.268390  2605 consensus_queue.cc:237] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:09.323794 29084 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:09.326097 29084 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:09.327574 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:09.318056 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:09.328151 29152 raft_consensus.cc:3055] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:09.328293 29715 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:09.328528 29152 raft_consensus.cc:740] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:09.329346 29152 consensus_queue.cc:260] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:09.330387 29152 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:09.331214 29007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
I20260709 13:13:09.331760 29007 raft_consensus.cc:3060] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:09.340495 29715 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:13:09.341681 28976 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:09.345273 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:09.345906 29157 raft_consensus.cc:3055] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:09.346151 29157 raft_consensus.cc:740] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:09.346678 29157 consensus_queue.cc:260] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:09.347597 29007 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:09.348302 29157 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:09.349234 29048 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:09.350476  2574 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:09.353276 29152 raft_consensus.cc:2468] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:09.357496  2574 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:09.361055  2574 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:09.366909  2626 ts_tablet_manager.cc:933] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:09.370440 29157 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 13.
I20260709 13:13:09.354523  2607 raft_consensus.cc:2804] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:09.388600  2607 raft_consensus.cc:697] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:09.389487  2607 consensus_queue.cc:237] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:09.408613 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 50905.4us
I20260709 13:13:09.414465 28904 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 12 to 13, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 13 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:09.426769  2626 tablet_copy_client.cc:323] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:13:09.429486 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet 6a08e186ea0749ae979329103591a933 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:13:09.430241 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet 6a08e186ea0749ae979329103591a933 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-6a08e186ea0749ae979329103591a933
I20260709 13:13:09.444036 29169 tablet_copy_source_session.cc:215] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:13:09.450054  2626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a08e186ea0749ae979329103591a933. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:09.484438  2585 log.cc:927] Time spent T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:13:09.674197 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 52490.3us
W20260709 13:13:09.675604 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 54937.3us
I20260709 13:13:09.682902  2647 raft_consensus.cc:993] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:09.710774 29006 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:13:09.721768 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 101710us
I20260709 13:13:09.723762  2650 ts_tablet_manager.cc:933] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:09.726172  2650 tablet_copy_client.cc:323] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
I20260709 13:13:09.727995 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet e1a1cae5d4eb46b288c03a3675d24209 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:13:09.728591 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet e1a1cae5d4eb46b288c03a3675d24209 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-e1a1cae5d4eb46b288c03a3675d24209
I20260709 13:13:09.739668 29095 tablet_copy_source_session.cc:215] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:13:09.733916  2544 raft_consensus.cc:1081] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
W20260709 13:13:09.768380  2635 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: Append to log took a long time: real 0.178s	user 0.003s	sys 0.000s
W20260709 13:13:09.787343 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 93388.5us
I20260709 13:13:09.791946 29010 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a08e186ea0749ae979329103591a933"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:09.792523 29010 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:09.792837 29010 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:09.789716 29003 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:09.819492 29010 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:13:09.826484 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 93632.8us
I20260709 13:13:09.833140 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:09.833683 29084 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:09.836313 29007 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 20 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:13:09.846253 29010 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:09.852500 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:09.853003  2642 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:13:09.857636  2650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a1cae5d4eb46b288c03a3675d24209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:09.853122 29157 raft_consensus.cc:3055] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:09.858999 29157 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:09.859802 29157 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:09.861078 29157 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:09.861794 29156 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:09.875583 29084 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:13:09.881564  2607 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:09.881592 29157 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:13:09.896618 28975 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:09.906232  2626 tablet_copy_client.cc:806] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:09.906910  2626 tablet_copy_client.cc:670] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:13:09.922441  2639 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:09.923100  2639 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:09.939291  2626 tablet_copy_client.cc:538] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:13:09.951331  2626 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:13:09.953260 29153 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:09.954437 29157 raft_consensus.cc:1275] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 20 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:13:09.974917  2646 raft_consensus.cc:993] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:09.975586  2577 raft_consensus.cc:1081] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:09.981925 29004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:09.982638 29004 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:09.983089 29004 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:09.987771 29002 raft_consensus.cc:1275] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:09.993428  2664 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:09.997556 28904 catalog_manager.cc:5697] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 19 to 20, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 20 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:10.002897  2645 consensus_queue.cc:1048] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:09.999514 28904 catalog_manager.cc:5697] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 15 to 16, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 16 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:10.007131 29711 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:10.013932  2625 consensus_queue.cc:1048] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:10.013947  2574 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:10.020934 29004 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:10.034092 29004 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:10.034363 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:10.035218 29157 raft_consensus.cc:3055] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:10.035579 29157 raft_consensus.cc:740] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:10.057328 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:10.058308 29077 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:10.063134 29157 consensus_queue.cc:260] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:09.924103  2639 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:10.107425 29077 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
W20260709 13:13:10.149714 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 95448.8us
I20260709 13:13:10.154107 28976 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:10.155678 29153 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:10.192662  2666 ts_tablet_manager.cc:933] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:10.203809 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 49808.7us
I20260709 13:13:10.237573  2649 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:10.244901  2675 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 13:13:10.255473  2667 log.cc:927] Time spent T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:13:10.257125  2649 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:10.258067  2649 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:10.330574  2650 tablet_copy_client.cc:806] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:10.331231  2650 tablet_copy_client.cc:670] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:13:10.333366 29078 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:10.378201  2574 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:13:10.365474 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:10.391976  2669 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
W20260709 13:13:10.379925  2674 log.cc:927] Time spent T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:13:10.453349 29157 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:10.471489 29157 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 14.
I20260709 13:13:10.520706  2666 tablet_copy_client.cc:323] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
W20260709 13:13:10.410032 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 86640.9us
W20260709 13:13:10.572007 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 234992us
I20260709 13:13:10.603138 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet fa4501421f5146e4a7b79fd8697f8f57 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:13:10.604210 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet fa4501421f5146e4a7b79fd8697f8f57 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-fa4501421f5146e4a7b79fd8697f8f57
I20260709 13:13:10.562723 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:10.680455 28976 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 824fc688db264856b025a29c651eec78. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:10.686614  2639 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:10.751379 29169 tablet_copy_source_session.cc:215] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:13:10.752341 29157 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:10.780314 29152 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:10.782465  2666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa4501421f5146e4a7b79fd8697f8f57. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:10.790434 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 78577.3us
I20260709 13:13:10.842654 29077 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:10.849869  2639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:40366 (request call id 0) took 1142 ms. Trace:
I20260709 13:13:10.850710  2639 rpcz_store.cc:276] 0709 13:13:09.707861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.708512 (+   651us) service_pool.cc:224] Handling call
0709 13:13:09.720300 (+ 11788us) log.cc:844] Serialized 499 byte log entry
0709 13:13:10.118154 (+397854us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b5400147ba0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157ba142 00007f411579aeaa 00007f41157a1d53 00007f41157a1cd9 00007f41157a1c69 00007f41157a1c31 00007f41157a0dfd 000000000038f354 000000000038f199 00007f4112754475
0709 13:13:10.120583 (+  2429us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b5400147ba0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157ba142 00007f411579aeaa 00007f41157a1d53 00007f41157a1cd9 00007f41157a1c69 00007f41157a1c31 00007f41157a0dfd 000000000038f354 000000000038f199 00007f4112754475
0709 13:13:10.122111 (+  1528us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b5400147ba0. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41157aa571 00007f41157c1b26 00007f41157bb49a 00007f41157ba714 00007f411579aeaa 00007f41157a1d53 00007f41157a1cd9 00007f41157a1c69 00007f41157a1c31 00007f41157a0dfd 000000000038f354
0709 13:13:10.426973 (+304862us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b4c00436a80. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f4113b0da20 00007f4113b1ea29 00007f4113b1f13f 00007f4113ae32ab 00007f4113b10f3c 00007f4113b113fe 00007f4113d9e50b 00007f411579d381 00007f411579b26b 00007f41157a1d53 00007f41157a1cd9
0709 13:13:10.849681 (+422708us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21487,"raft.queue_time_us":58092,"raft.run_cpu_time_us":14606,"raft.run_wall_time_us":515197,"spinlock_wait_cycles":158824704,"thread_start_us":298266,"threads_started":1,"wal-append.queue_time_us":398016}
I20260709 13:13:10.902820  2530 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:13:10.917683 29078 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:10.919951  2691 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:10.928543  2650 tablet_copy_client.cc:538] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:13:10.945369 29079 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:10.997184  2691 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:11.022855  2650 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:13:11.061830 29151 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:13:11.105222 28976 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:11.120424 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 45749.4us
I20260709 13:13:11.137111  2664 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:11.151582  2666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:49414 (request call id 890) took 1267 ms. Trace:
I20260709 13:13:11.152083  2666 rpcz_store.cc:276] 0709 13:13:09.883849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.884008 (+   159us) service_pool.cc:224] Handling call
0709 13:13:10.193183 (+309175us) ts_tablet_manager.cc:934] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
0709 13:13:10.445548 (+252365us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007f4112722343 00007f4112722240 00007f41120b11d4 00007f411bdef304 00007f411bdef299 00007f41125dcdb1 00007f411267ac85 00007f411267b6a2 00007f41153ed1db 00007f41153e8aaf 00007f41127370ac 00007f411b591ab9 00007f411275369b 00007f4112750caa 00007f4112750bd2 00007f411a7b5131
0709 13:13:10.768785 (+323237us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:13:10.926701 (+157916us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:10.926825 (+   124us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:13:11.144672 (+217847us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:13:11.145007 (+   335us) tablet_replica.cc:206] Creating consensus instance
0709 13:13:11.151279 (+  6272us) ts_tablet_manager.cc:989] Replica registered.
0709 13:13:11.151470 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":263,"spinlock_wait_cycles":46151552,"tablet-copy.queue_time_us":216398,"thread_start_us":398317,"threads_started":2}
I20260709 13:13:11.153860  2666 tablet_copy_client.cc:806] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:11.154462  2666 tablet_copy_client.cc:670] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:13:11.208163  2666 tablet_copy_client.cc:538] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:13:11.193179 28900 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 17 to 18, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 18 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:11.249035 28900 catalog_manager.cc:5697] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 13 to 14, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 14 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:11.308501  2708 ts_tablet_manager.cc:933] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
I20260709 13:13:11.442049  2666 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:13:11.451455  2626 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:11.453873  2626 tablet_bootstrap.cc:492] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:13:11.454839  2626 ts_tablet_manager.cc:1403] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 1.504s	user 0.066s	sys 0.021s
I20260709 13:13:11.506922  2708 tablet_copy_client.cc:323] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:13:11.510556  2626 raft_consensus.cc:359] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:11.511492 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet 824fc688db264856b025a29c651eec78 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:13:11.512177 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet 824fc688db264856b025a29c651eec78 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-824fc688db264856b025a29c651eec78
I20260709 13:13:11.511494  2626 raft_consensus.cc:740] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:13:11.515434  2626 consensus_queue.cc:260] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:11.518664  2626 ts_tablet_manager.cc:1434] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:13:11.523550 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-6a08e186ea0749ae979329103591a933 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:13:11.524099 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-6a08e186ea0749ae979329103591a933 on tablet 6a08e186ea0749ae979329103591a933 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:11.539151 29020 tablet_copy_source_session.cc:215] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:13:11.542810  2708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824fc688db264856b025a29c651eec78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:11.594854 28905 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 15 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:11.610010  2708 tablet_copy_client.cc:806] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:11.610864  2708 tablet_copy_client.cc:670] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:13:11.620837  2708 tablet_copy_client.cc:538] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:13:11.625612  2650 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:11.636680  2650 tablet_bootstrap.cc:492] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:13:11.637425  2650 ts_tablet_manager.cc:1403] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.615s	user 0.030s	sys 0.018s
I20260709 13:13:11.653780  2650 raft_consensus.cc:359] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:11.654675  2650 raft_consensus.cc:740] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:13:11.655346  2650 consensus_queue.cc:260] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:11.752627  2650 ts_tablet_manager.cc:1434] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.115s	user 0.007s	sys 0.002s
I20260709 13:13:11.762471  2708 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:13:11.768504 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-e1a1cae5d4eb46b288c03a3675d24209 received from {username='slave'} at 127.0.0.1:39900
I20260709 13:13:11.769022 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-e1a1cae5d4eb46b288c03a3675d24209 on tablet e1a1cae5d4eb46b288c03a3675d24209 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:11.792131  2666 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=2 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:11.793145  2666 tablet_bootstrap.cc:492] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:13:11.794070  2666 ts_tablet_manager.cc:1403] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.352s	user 0.048s	sys 0.017s
I20260709 13:13:11.800210 29715 raft_consensus.cc:1217] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Deduplicated request from leader. Original: 17.10->[17.11-17.11]   Dedup: 17.11->[]
I20260709 13:13:11.804762  2666 raft_consensus.cc:359] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:11.805686  2666 raft_consensus.cc:740] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:13:11.814368  2666 consensus_queue.cc:260] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:11.818703  2666 ts_tablet_manager.cc:1434] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:13:11.825752 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-fa4501421f5146e4a7b79fd8697f8f57 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:13:11.826236 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-fa4501421f5146e4a7b79fd8697f8f57 on tablet fa4501421f5146e4a7b79fd8697f8f57 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:11.850485 29715 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Advancing to term 18
I20260709 13:13:11.990154  2708 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:11.996495  2708 tablet_bootstrap.cc:492] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:13:12.000106  2708 ts_tablet_manager.cc:1403] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.235s	user 0.039s	sys 0.038s
I20260709 13:13:12.004174  2708 raft_consensus.cc:359] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:12.005417  2708 raft_consensus.cc:740] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:13:12.023593  2708 consensus_queue.cc:260] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:12.045971  2708 ts_tablet_manager.cc:1434] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 13:13:12.060611 29020 tablet_copy_service.cc:342] P 06ae4197fd784525b695a9b16c8094ea: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-824fc688db264856b025a29c651eec78 received from {username='slave'} at 127.0.0.1:33750
I20260709 13:13:12.061156 29020 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-824fc688db264856b025a29c651eec78 on tablet 824fc688db264856b025a29c651eec78 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:12.193444 29715 raft_consensus.cc:1217] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 LEARNER]: Deduplicated request from leader. Original: 15.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 13:13:12.252490  2680 raft_consensus.cc:1064] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:13:12.255488  2680 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:12.296058 29003 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:13:12.297857 29153 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:13:12.300824  2561 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:12.302229 29715 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:13:12.305202  2561 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:12.308854  2561 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:12.315912 29711 raft_consensus.cc:1217] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 LEARNER]: Deduplicated request from leader. Original: 13.8->[13.9-13.9]   Dedup: 13.9->[]
I20260709 13:13:12.356377  2680 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.395380 29151 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.457598 28905 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 11 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.493403  2664 raft_consensus.cc:1064] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:13:12.501806  2664 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 13, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:12.512220 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 62485.6us
I20260709 13:13:12.514564 29711 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.539338 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:12.554970 29002 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.661576 29153 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:13:12.671886  2530 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:12.675866 29084 consensus_queue.cc:237] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:12.680630 29077 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:13:12.680649 29711 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:13:12.682857  2691 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:12.692260  2732 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:12.729252 29153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641"
dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
mode: GRACEFUL
new_leader_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:52580
I20260709 13:13:12.730008 29153 raft_consensus.cc:606] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Received request to transfer leadership to TS 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:12.731144 29076 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 13 to 15, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.729939  2691 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Committing config change with OpId 18.15: config changed from index 13 to 15, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.735441 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:13:12.737426 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:12.747347 29711 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:12.749733  2609 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:12.750492 28901 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 13 to 15, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.757894  2729 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.764289 29002 raft_consensus.cc:1275] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:12.764776 29711 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.765540  2680 consensus_queue.cc:1048] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:12.773685 29006 raft_consensus.cc:2955] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.793582 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e1a1cae5d4eb46b288c03a3675d24209 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:13:12.800381 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 130265us
I20260709 13:13:12.802428 28901 catalog_manager.cc:5697] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 12 to 13, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:12.838672  2631 raft_consensus.cc:1064] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:13:12.845989  2631 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:12.851365 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a1cae5d4eb46b288c03a3675d24209 with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:57656
W20260709 13:13:12.858210 28975 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.27.63.126:43467, user_credentials={real_user=slave}} blocked reactor thread for 50701.1us
I20260709 13:13:12.920394 29151 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 13 to 15, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:13:12.925242 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 59084.6us
I20260709 13:13:12.954670  2739 tablet_replica.cc:333] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:12.955876  2739 raft_consensus.cc:2243] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:12.956714  2739 raft_consensus.cc:2272] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:12.967233  2732 raft_consensus.cc:1064] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
W20260709 13:13:13.002017 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 127856us
I20260709 13:13:13.004537  2732 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:13.014880 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 79468.7us
I20260709 13:13:13.019423  2739 ts_tablet_manager.cc:1916] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:13:13.046206 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 71540.6us
I20260709 13:13:13.046962 29002 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:13.048517 29711 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:13.050139  2631 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:13.055195  2744 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:13:13.060370 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 67300.7us
I20260709 13:13:13.073959 29076 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:13:13.075043 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 71591.1us
I20260709 13:13:13.089473  2747 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:13:13.192128 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51248.4us
W20260709 13:13:13.203390 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 71589.9us
I20260709 13:13:13.250062 29717 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 13 to 15, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.292315  2748 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index 9 to 10, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.306706 29084 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:13.337417 29715 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 LEARNER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:13:13.350782 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 130113us
I20260709 13:13:13.360024 29711 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.377559 29151 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:13.384680  2761 raft_consensus.cc:993] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: : Instructing follower 06ae4197fd784525b695a9b16c8094ea to start an election
I20260709 13:13:13.385329  2517 raft_consensus.cc:1081] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Signalling peer 06ae4197fd784525b695a9b16c8094ea to start an election
W20260709 13:13:13.388832  2755 log.cc:927] Time spent T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Append to log took a long time: real 0.090s	user 0.000s	sys 0.003s
I20260709 13:13:13.409898 29007 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641"
dest_uuid: "06ae4197fd784525b695a9b16c8094ea"
 from {username='slave'} at 127.0.0.1:34936
I20260709 13:13:13.483912 29007 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:13.484380 29007 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:13.353458  2753 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:13:13.491017  2691 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:13:13.496191 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 53395.3us
I20260709 13:13:13.496630  2743 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:13:13.485689 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 60259.1us
W20260709 13:13:13.568248 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 178953us
I20260709 13:13:13.581113 29007 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:13.587299 29004 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.588788 29007 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:13.602919  2739 ts_tablet_manager.cc:1929] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.12
I20260709 13:13:13.603600  2739 log.cc:1199] T e1a1cae5d4eb46b288c03a3675d24209 P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/e1a1cae5d4eb46b288c03a3675d24209
I20260709 13:13:13.609782 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:13.610522 29084 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:13.612385 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet e1a1cae5d4eb46b288c03a3675d24209 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:13.621297 29003 raft_consensus.cc:1240] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Rejecting Update request from peer 243d4ae4a4ae48b2979e32874472d256 for earlier term 17. Current term is 18. Ops: []
W20260709 13:13:13.624334 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51365.8us
I20260709 13:13:13.634014 29003 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:13.636482 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:13.637127 29154 raft_consensus.cc:3055] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:13.637441 29154 raft_consensus.cc:740] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:13.638603  2765 consensus_queue.cc:1059] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:13.646744 28906 catalog_manager.cc:5697] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 9 to 10, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.650583 29084 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
I20260709 13:13:13.668411 29078 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.674256 29154 consensus_queue.cc:260] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:13.682808 29154 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:13.675976 29716 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:13:13.704550 29154 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ae4197fd784525b695a9b16c8094ea in term 18.
W20260709 13:13:13.705406  2765 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:13:13.710079 28975 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256; no voters: 
I20260709 13:13:13.712011  2764 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:13.712558  2764 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Becoming Leader. State: Replica: 06ae4197fd784525b695a9b16c8094ea, State: Running, Role: LEADER
I20260709 13:13:13.713450  2764 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:13.712093  2691 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.718677 29151 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:13:13.724080  2753 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:13:13.722621 29716 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.735874  2743 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:13:13.740032 29717 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Committing config change with OpId 18.16: config changed from index 15 to 16, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.749493 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:13.738334  2753 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Committing config change with OpId 18.16: config changed from index 15 to 16, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.771912 29085 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.800294 29154 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.806085 29153 raft_consensus.cc:2955] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Committing config change with OpId 18.16: config changed from index 15 to 16, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.815637 28892 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6a08e186ea0749ae979329103591a933 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:13:13.866328 28905 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.937224 28904 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 15 to 16, VOTER befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) evicted. New cstate: current_term: 18 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:13.976434 28906 catalog_manager.cc:5697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: term changed from 17 to 18, leader changed from 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) to 06ae4197fd784525b695a9b16c8094ea (127.27.63.65). New cstate: current_term: 18 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:13.988910 29067 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a08e186ea0749ae979329103591a933 with delete_type TABLET_DATA_TOMBSTONED (TS befda4a6c8344c26b0f139e4e1686b3b not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:44418
I20260709 13:13:14.009449 29154 consensus_queue.cc:237] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:14.016943 29716 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:14.016937 29085 raft_consensus.cc:1275] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:14.019790  2765 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:14.023955  2765 consensus_queue.cc:1048] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:14.030560 29003 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:14.034888  2758 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:14.045449  2778 tablet_replica.cc:333] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:14.044736 29716 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:14.048308 28894 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fa4501421f5146e4a7b79fd8697f8f57 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:13:14.061022 29076 raft_consensus.cc:2955] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:14.046667  2778 raft_consensus.cc:2243] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:14.070366  2778 raft_consensus.cc:2272] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:14.082037 28906 catalog_manager.cc:5697] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 10 to 11, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 13 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:14.100544  2778 ts_tablet_manager.cc:1916] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:13:14.104455 28996 tablet_service.cc:1558] Processing DeleteTablet for tablet fa4501421f5146e4a7b79fd8697f8f57 with delete_type TABLET_DATA_TOMBSTONED (TS 06ae4197fd784525b695a9b16c8094ea not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:34914
I20260709 13:13:14.126600 29716 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:14.126766 29085 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:14.128509  2743 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:14.129536  2753 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:14.135558  2779 tablet_replica.cc:333] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:14.137872  2779 raft_consensus.cc:2243] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:14.138849  2779 raft_consensus.cc:2272] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:14.141733  2743 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:14.145191  2779 ts_tablet_manager.cc:1916] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:13:14.150708 29076 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:14.165295 29716 raft_consensus.cc:2955] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:14.182083 28892 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 824fc688db264856b025a29c651eec78 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:13:14.194283  2778 ts_tablet_manager.cc:1929] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.15
I20260709 13:13:14.196235  2778 log.cc:1199] T 6a08e186ea0749ae979329103591a933 P befda4a6c8344c26b0f139e4e1686b3b: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/wals/6a08e186ea0749ae979329103591a933
I20260709 13:13:14.197896 28901 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index 11 to 12, VOTER 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67) evicted. New cstate: current_term: 15 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:14.214197 29147 tablet_service.cc:1558] Processing DeleteTablet for tablet 824fc688db264856b025a29c651eec78 with delete_type TABLET_DATA_TOMBSTONED (TS 243d4ae4a4ae48b2979e32874472d256 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:57656
I20260709 13:13:14.213779 28895 catalog_manager.cc:5028] TS befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): tablet 6a08e186ea0749ae979329103591a933 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:14.259253  2780 tablet_replica.cc:333] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:14.265136  2779 ts_tablet_manager.cc:1929] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.10
I20260709 13:13:14.266912  2779 log.cc:1199] T fa4501421f5146e4a7b79fd8697f8f57 P 06ae4197fd784525b695a9b16c8094ea: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/wals/fa4501421f5146e4a7b79fd8697f8f57
I20260709 13:13:14.270474 28892 catalog_manager.cc:5028] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): tablet fa4501421f5146e4a7b79fd8697f8f57 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:14.272876  2780 raft_consensus.cc:2243] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:14.276405  2780 raft_consensus.cc:2272] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:14.287325  2780 ts_tablet_manager.cc:1916] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:13:14.298689 29076 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:14.300386  2691 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:14.323493  2780 ts_tablet_manager.cc:1929] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 13:13:14.328249  2780 log.cc:1199] T 824fc688db264856b025a29c651eec78 P 243d4ae4a4ae48b2979e32874472d256: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/wals/824fc688db264856b025a29c651eec78
I20260709 13:13:14.329967 28894 catalog_manager.cc:5028] TS 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): tablet 824fc688db264856b025a29c651eec78 (table test-workload [id=db7088a4cae2474f8ae2dd254e96e144]) successfully deleted
I20260709 13:13:14.343117 29153 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:14.346341  2753 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:14.904862 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:15.006248 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:13:15.018532 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:15.427983 29151 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:13:15.437414 29125 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 640d00aba1b541e4abb90c63e6b8720d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:15.442849 29085 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:15.444474  2765 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:15.456812 29003 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:15.459939  2765 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:15.484863  2765 raft_consensus.cc:2955] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.489972 29076 raft_consensus.cc:2955] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.494887 29003 raft_consensus.cc:2955] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.510655 28906 catalog_manager.cc:5697] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
W20260709 13:13:15.570519 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 62362us
I20260709 13:13:15.592679 29003 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:15.647372 29085 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:13:15.649276 28976 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf1e3918423546bcafc9359aca2ca811. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:15.650177  2741 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:15.650887 29151 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer 06ae4197fd784525b695a9b16c8094ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:15.652544  2741 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:15.675189  2741 raft_consensus.cc:2955] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.688097 29076 raft_consensus.cc:2955] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.689924 29151 raft_consensus.cc:2955] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.703662 29154 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:15.713954 28901 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 10, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 16 leader_uuid: "06ae4197fd784525b695a9b16c8094ea" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:15.802693 29004 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:15.805918 29076 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:15.806578  2798 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:13:15.817040 29125 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d012d02d5bf34cbf8f4451c054b36f63. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:15.818332  2798 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:15.857645 29004 raft_consensus.cc:2955] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Committing config change with OpId 15.8: config changed from index -1 to 8, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.854938  2798 raft_consensus.cc:2955] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Committing config change with OpId 15.8: config changed from index -1 to 8, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.895890 29084 raft_consensus.cc:2955] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Committing config change with OpId 15.8: config changed from index -1 to 8, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.905186 29076 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:15.905324 28904 catalog_manager.cc:5697] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea reported cstate change: config changed from index -1 to 8, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
W20260709 13:13:15.930187 29051 consensus_peers.cc:597] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bbf1db4b075436695fd521f5a5b2e9d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:15.931298 29151 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:15.932873 29004 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:15.933048  2728 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:15.934377  2729 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:15.990259 29004 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:15.988682  2728 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:16.003010 29151 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:16.022779 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:16.031054 28904 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index -1 to 12, NON_VOTER ca45420e033847489ad989c1c6eed3ef (127.27.63.68) added. New cstate: current_term: 16 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } } }
I20260709 13:13:16.073709  2814 ts_tablet_manager.cc:933] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:16.077765  2812 ts_tablet_manager.cc:933] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941)
W20260709 13:13:16.159763 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 57841.9us
I20260709 13:13:16.180338  2812 tablet_copy_client.cc:323] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.65:43941
I20260709 13:13:16.199461 29020 tablet_copy_service.cc:140] P 06ae4197fd784525b695a9b16c8094ea: Received BeginTabletCopySession request for tablet bf1e3918423546bcafc9359aca2ca811 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:33750)
I20260709 13:13:16.200078 29020 tablet_copy_service.cc:161] P 06ae4197fd784525b695a9b16c8094ea: Beginning new tablet copy session on tablet bf1e3918423546bcafc9359aca2ca811 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:33750: session id = ca45420e033847489ad989c1c6eed3ef-bf1e3918423546bcafc9359aca2ca811
I20260709 13:13:16.204186  2814 tablet_copy_client.cc:323] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
I20260709 13:13:16.206826 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet 640d00aba1b541e4abb90c63e6b8720d from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:13:16.207458 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet 640d00aba1b541e4abb90c63e6b8720d from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-640d00aba1b541e4abb90c63e6b8720d
I20260709 13:13:16.226809 29020 tablet_copy_source_session.cc:215] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:13:16.282441 29169 tablet_copy_source_session.cc:215] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:13:16.315006  2814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 640d00aba1b541e4abb90c63e6b8720d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:16.321233  2812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf1e3918423546bcafc9359aca2ca811. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:16.342826 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 51029.3us
I20260709 13:13:16.371861 27901 tablet_server.cc:179] TabletServer@127.27.63.65:0 shutting down...
W20260709 13:13:16.382513 29122 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
W20260709 13:13:16.442185 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:16.442791  2825 ts_tablet_manager.cc:933] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:16.445993 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.448227 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.451829 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.453398 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 61754.2us
W20260709 13:13:16.454133 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.454759 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 60386.8us
W20260709 13:13:16.475490 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.473626 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:16.476918  2812 tablet_copy_client.cc:806] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:16.475423  2814 tablet_copy_client.cc:806] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:16.491183  2814 tablet_copy_client.cc:670] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:13:16.488083  2812 tablet_copy_client.cc:670] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
W20260709 13:13:16.538825 28975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 47098us
W20260709 13:13:16.549613 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:16.564924  2814 tablet_copy_client.cc:538] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:13:16.556981  2825 tablet_copy_client.cc:323] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.67:40957
W20260709 13:13:16.570483 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 94488.1us
W20260709 13:13:16.616865 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 45346.7us
I20260709 13:13:16.628209 29169 tablet_copy_service.cc:140] P 243d4ae4a4ae48b2979e32874472d256: Received BeginTabletCopySession request for tablet d012d02d5bf34cbf8f4451c054b36f63 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:32836)
I20260709 13:13:16.628871 29169 tablet_copy_service.cc:161] P 243d4ae4a4ae48b2979e32874472d256: Beginning new tablet copy session on tablet d012d02d5bf34cbf8f4451c054b36f63 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:32836: session id = ca45420e033847489ad989c1c6eed3ef-d012d02d5bf34cbf8f4451c054b36f63
W20260709 13:13:16.620074 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.634202 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:16.663169  2814 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:13:16.683642 29169 tablet_copy_source_session.cc:215] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:13:16.702785 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:16.750797  2825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d012d02d5bf34cbf8f4451c054b36f63. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:16.753718 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.772490 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:16.772481  2832 ts_tablet_manager.cc:933] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:16.792682 27901 tablet_copy_service.cc:362] P 06ae4197fd784525b695a9b16c8094ea: Destroying tablet copy session ca45420e033847489ad989c1c6eed3ef-bf1e3918423546bcafc9359aca2ca811 due to service shutdown
I20260709 13:13:16.793165 27901 tablet_copy_service.cc:434] P 06ae4197fd784525b695a9b16c8094ea: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-bf1e3918423546bcafc9359aca2ca811 on tablet bf1e3918423546bcafc9359aca2ca811 with peer ca45420e033847489ad989c1c6eed3ef
W20260709 13:13:16.812779 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.837390 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 83102.7us
W20260709 13:13:16.839051 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.839632 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.853871 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.854378 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80298.4us
W20260709 13:13:16.839741 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.874555 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.875754 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 48150.8us
W20260709 13:13:16.892094 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.891564 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.896653 29048 consensus_peers.cc:597] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.897183 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.897640 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.918355 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.932930 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.927594 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:16.921128  2832 tablet_copy_client.cc:323] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
W20260709 13:13:16.968106 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.969295 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.977098 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:16.977695 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:17.037482 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet 2bbf1db4b075436695fd521f5a5b2e9d from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:13:17.038144 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet 2bbf1db4b075436695fd521f5a5b2e9d from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-2bbf1db4b075436695fd521f5a5b2e9d
W20260709 13:13:17.041257 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:17.044237 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:17.260679 29095 tablet_copy_source_session.cc:215] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:13:17.288228 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 178144us
I20260709 13:13:17.335434  2832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bbf1db4b075436695fd521f5a5b2e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:17.402020  2825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:49414 (request call id 947) took 1068 ms. Trace:
I20260709 13:13:17.461021  2842 raft_consensus.cc:493] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.461632  2842 raft_consensus.cc:515] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:17.429919  2823 raft_consensus.cc:493] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
W20260709 13:13:17.497820 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 108399us
I20260709 13:13:17.476321  2741 raft_consensus.cc:493] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.496243  2753 raft_consensus.cc:493] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.518375  2823 raft_consensus.cc:515] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.554405  2823 leader_election.cc:290] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.554914  2823 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.555371  2823 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.557343  2823 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.429520  2825 rpcz_store.cc:276] 0709 13:13:16.333261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.333402 (+   141us) service_pool.cc:224] Handling call
0709 13:13:16.443179 (+109777us) ts_tablet_manager.cc:934] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
0709 13:13:16.740742 (+297563us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:13:16.912934 (+172192us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:16.916926 (+  3992us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:13:17.309646 (+392720us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:13:17.332615 (+ 22969us) tablet_replica.cc:206] Creating consensus instance
0709 13:13:17.388729 (+ 56114us) ts_tablet_manager.cc:989] Replica registered.
0709 13:13:17.394363 (+  5634us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":68939,"tablet-copy.queue_time_us":108893,"thread_start_us":86383,"threads_started":2}
I20260709 13:13:17.545483  2753 raft_consensus.cc:515] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.539815  2741 raft_consensus.cc:515] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.576613  2753 leader_election.cc:290] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.576633  2842 leader_election.cc:290] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.567090  2825 tablet_copy_client.cc:806] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:17.580540  2825 tablet_copy_client.cc:670] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:13:17.549485  2844 raft_consensus.cc:493] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:17.592753  2844 raft_consensus.cc:515] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.595487  2844 leader_election.cc:290] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.617341  2741 leader_election.cc:290] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:17.623824 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 133095us
I20260709 13:13:17.625877 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.629410 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.632673  2741 raft_consensus.cc:493] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.776815  2691 raft_consensus.cc:493] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:17.791106  2691 raft_consensus.cc:515] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.796253  2865 raft_consensus.cc:493] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.796846  2865 raft_consensus.cc:515] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.798496  2691 leader_election.cc:290] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.778800  2842 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.806845  2842 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.808841  2842 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.729285  2860 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.809463  2860 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.810436 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:17.780191  2827 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.812333  2827 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.812701 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:17.782435  2741 raft_consensus.cc:515] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:17.784809 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 138108us
I20260709 13:13:17.785312  2823 raft_consensus.cc:493] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:17.814586  2827 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.811914  2860 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.825793 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:13:17.827200  2741 leader_election.cc:290] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.826975  2823 raft_consensus.cc:515] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.829227  2823 leader_election.cc:290] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.829794 27901 tablet_replica.cc:333] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:17.814877 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:17.786950  2852 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.830751 27901 raft_consensus.cc:2243] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:17.830564  2852 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.800225  2865 leader_election.cc:290] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.801827  2861 raft_consensus.cc:493] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.801833 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c5d53651b5426780911903843f0bc3" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:17.831523 27901 raft_consensus.cc:2272] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:17.786391  2850 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.831705  2861 raft_consensus.cc:515] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.832429  2850 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.832933  2852 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.833927  2861 leader_election.cc:290] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.834535  2850 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.835690  2865 raft_consensus.cc:493] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.836369  2865 raft_consensus.cc:515] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.842363  2850 raft_consensus.cc:493] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:17.843082 27901 tablet_replica.cc:333] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:17.844606  2861 raft_consensus.cc:493] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:17.857040  2850 raft_consensus.cc:515] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.857266  2861 raft_consensus.cc:515] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.859339  2850 leader_election.cc:290] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.851841  2852 raft_consensus.cc:493] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:17.859858  2850 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.860636  2852 raft_consensus.cc:515] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.860841  2850 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.856493  2741 raft_consensus.cc:493] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:17.862206  2741 raft_consensus.cc:515] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:17.862845  2852 leader_election.cc:290] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.859949  2861 leader_election.cc:290] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.864209  2861 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.864795  2861 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.865981 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "828bc1c562f941c89bdcc43cf79e5ece" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.866900  2861 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.864814 27901 raft_consensus.cc:2243] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:17.869508 27901 raft_consensus.cc:2272] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:17.871248 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 86754.9us
I20260709 13:13:17.865353  2741 leader_election.cc:290] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.874974  2865 leader_election.cc:290] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.878378  2823 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.879113 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d012d02d5bf34cbf8f4451c054b36f63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.777979  2824 raft_consensus.cc:493] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.880127  2824 raft_consensus.cc:515] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:17.881338 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.890129 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.863276  2850 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:17.894394  2832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:47384 (request call id 1308) took 1369 ms. Trace:
I20260709 13:13:17.878975  2823 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.901520  2832 rpcz_store.cc:276] 0709 13:13:16.525472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.525656 (+   184us) service_pool.cc:224] Handling call
0709 13:13:16.778026 (+252370us) ts_tablet_manager.cc:934] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
0709 13:13:17.312945 (+534919us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:13:17.639332 (+326387us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:17.643270 (+  3938us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:13:17.842613 (+199343us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:13:17.842883 (+   270us) tablet_replica.cc:206] Creating consensus instance
0709 13:13:17.894272 (+ 51389us) ts_tablet_manager.cc:989] Replica registered.
0709 13:13:17.894333 (+    61us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":8851,"tablet-copy.queue_time_us":209848,"thread_start_us":274627,"threads_started":2}
I20260709 13:13:17.915150  2865 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.915457  2832 tablet_copy_client.cc:806] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:17.916004  2832 tablet_copy_client.cc:670] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
I20260709 13:13:17.915675  2865 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.918203  2865 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.905769 27901 tablet_replica.cc:333] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:17.919894 27901 raft_consensus.cc:2243] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:17.921868 29157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.908556 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97f68627cfa4585b418a3c090e44c14" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:17.910912 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2185927c34434da38aa9e7776e483b1b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:17.914654  2850 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.922026 29151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.939167  2850 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.941702  2850 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.907600  2824 leader_election.cc:290] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.923005 29154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.923187  2865 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:17.923743  2844 raft_consensus.cc:493] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.924090 29153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.943140  2844 raft_consensus.cc:515] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.924329  2741 raft_consensus.cc:493] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:17.945434  2741 raft_consensus.cc:515] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:17.947741  2865 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:13:17.924970  2861 raft_consensus.cc:670] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 13:13:17.925791 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.925833 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.926505 27901 raft_consensus.cc:2272] T 8d989a55ed204649a0440142ce55e5ef P 06ae4197fd784525b695a9b16c8094ea [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:17.926774 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.927649 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.929171 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.931962 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.957881 27901 tablet_replica.cc:333] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:17.933826  2823 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.958755 27901 raft_consensus.cc:2243] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:17.968492 27901 raft_consensus.cc:2272] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:17.975844 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 88504us
I20260709 13:13:17.977479 27901 tablet_replica.cc:333] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:17.979013 27901 raft_consensus.cc:2243] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:17.980176  2844 leader_election.cc:290] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.982589 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.987812 27901 raft_consensus.cc:2272] T 824fc688db264856b025a29c651eec78 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:17.982858 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.979305 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e583be49bed64dd2996afcdcffa95120" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:17.986044  2865 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.982314 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:17.985349 28975 leader_election.cc:304] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:17.982529  2741 leader_election.cc:290] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:17.994563 28975 leader_election.cc:304] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:17.995188  2741 raft_consensus.cc:2749] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:17.995837  2741 raft_consensus.cc:2749] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:18.000792 28975 leader_election.cc:304] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.001453  2865 raft_consensus.cc:2749] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:18.010696 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.012120 29710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49241be91ca743cbac56eb24218ff775" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.012336 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.013681 29712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.015144 29711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342b8b8f6e7742c897bcd3495fecb00a" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.015641 29708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.015189 29709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.017302 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.018370 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.019569 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32aba72953748b4bb7a95efc76501ff" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.020929 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aa255ba50b343eca7717515ec37d959" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.021827 27901 tablet_replica.cc:333] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.022483 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00715172c7364d849f262c553c53e5fe" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.022812 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cac0f90a42148b5923aa84c7b7cf00b" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.025489  2825 tablet_copy_client.cc:538] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:13:18.029377 27901 raft_consensus.cc:2243] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.030439 27901 raft_consensus.cc:2272] T 6a08e186ea0749ae979329103591a933 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.032984 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.033576  2865 raft_consensus.cc:493] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:18.034142  2865 raft_consensus.cc:515] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:18.035166  2825 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:13:18.036252  2865 leader_election.cc:290] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:18.036635 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b9b21ce33904ed489b959d5aef0349f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.037793 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da4f9ce63d54a63a8b9de11ba485699" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.043556 28976 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.045122 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1db879e214974ab38bc94a40afcf6e1e" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.046506 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141925650a74de58026be545d874618" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.049700 27901 tablet_replica.cc:333] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.050719 27901 raft_consensus.cc:2243] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.051206  2865 raft_consensus.cc:2706] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Received pre-election callback for term 13 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 13:13:18.052323 28976 leader_election.cc:304] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.053078  2865 raft_consensus.cc:2749] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:18.054373 27901 raft_consensus.cc:2272] T f97f68627cfa4585b418a3c090e44c14 P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.058818 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:18.059092 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8e44f6f34a4291baf426298bc260e6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.060257 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dd7e7308fc43e0b0983412129aa471" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.060560 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d163c72e704bd0bd37c74d96761663" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.060859 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a1cae5d4eb46b288c03a3675d24209" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.062263 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:18.063638 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:18.064993 29159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:18.068416 27901 tablet_replica.cc:333] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.068818  2865 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:18.072675 28976 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.073966 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e87ad9e9a5439d890ebaf60f043eb6" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.074343  2741 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:18.073921 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df5a8a9dbd84e3bad34bf60478e7c98" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.071702  2865 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:18.076463 27901 raft_consensus.cc:2243] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.076593  2865 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:18.085279 27901 raft_consensus.cc:2272] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.085633  2814 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:18.086696 29078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb3976ea75a4cb3b4f543d842c2409c" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.087182 28976 leader_election.cc:304] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.087916 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcde0796c504aa0ad4a5e79df6a169d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.087940  2865 raft_consensus.cc:2749] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:18.088979 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e21ca94fafcc4923a671c355b710f10d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.089589 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640d00aba1b541e4abb90c63e6b8720d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.089779 29076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bf52cc065f943c9a83088bbeea5c794" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.090291  2814 tablet_bootstrap.cc:492] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:13:18.090375 28976 leader_election.cc:304] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.090958 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8340da85294490fb88f62f80393531f" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.093622 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e225d0308db94fb9ab850a0b06a63551" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.093976 27901 tablet_replica.cc:333] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.094889 27901 raft_consensus.cc:2243] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.095816 27901 raft_consensus.cc:2272] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.098512 27901 tablet_replica.cc:333] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.099328 27901 raft_consensus.cc:2243] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.099928 27901 raft_consensus.cc:2272] T 31c5d53651b5426780911903843f0bc3 P 06ae4197fd784525b695a9b16c8094ea [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.091156  2814 ts_tablet_manager.cc:1403] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 1.428s	user 0.051s	sys 0.046s
W20260709 13:13:18.091092 28976 raft_consensus.cc:2686] T c32aba72953748b4bb7a95efc76501ff P 06ae4197fd784525b695a9b16c8094ea: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:13:18.101275 28976 leader_election.cc:304] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.102782  2865 raft_consensus.cc:2749] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:18.103938 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.112265  2832 tablet_copy_client.cc:538] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:13:18.117398  2814 raft_consensus.cc:359] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:18.123270  2814 raft_consensus.cc:740] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:13:18.122095  2832 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:13:18.105489 28976 leader_election.cc:304] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.124135  2814 consensus_queue.cc:260] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:18.128772  2865 raft_consensus.cc:2749] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:18.130576 28976 leader_election.cc:304] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.131233  2865 raft_consensus.cc:2749] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:18.131636  2814 ts_tablet_manager.cc:1434] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:13:18.138567 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-640d00aba1b541e4abb90c63e6b8720d received from {username='slave'} at 127.0.0.1:32836
I20260709 13:13:18.139021 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-640d00aba1b541e4abb90c63e6b8720d on tablet 640d00aba1b541e4abb90c63e6b8720d with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.182993 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:18.132656 28976 leader_election.cc:304] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.134219 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafcee70c76341f2a36a461c97f2b3ba" candidate_uuid: "06ae4197fd784525b695a9b16c8094ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:18.185477 27901 tablet_replica.cc:333] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.186502 27901 raft_consensus.cc:2243] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:13:18.204227 28976 raft_consensus.cc:2686] T 00715172c7364d849f262c553c53e5fe P 06ae4197fd784525b695a9b16c8094ea: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:18.204794 28976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 72207.5us
I20260709 13:13:18.205456 28976 leader_election.cc:304] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.204788 27901 raft_consensus.cc:2272] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.206526  2865 raft_consensus.cc:2749] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:18.209348 27901 tablet_replica.cc:333] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.210122 27901 raft_consensus.cc:2243] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.211297 28976 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.217106  2865 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:18.218232 28976 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:18.227236 28976 raft_consensus.cc:2686] T 9b9b21ce33904ed489b959d5aef0349f P 06ae4197fd784525b695a9b16c8094ea: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:13:18.228153 28976 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.230590 28976 leader_election.cc:304] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.231120 27901 raft_consensus.cc:2272] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.231650 28976 leader_election.cc:304] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:13:18.234342 28976 raft_consensus.cc:2686] T c141925650a74de58026be545d874618 P 06ae4197fd784525b695a9b16c8094ea: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:13:18.235371 28976 leader_election.cc:304] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.236402 28976 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.237250 28976 leader_election.cc:304] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.237849  2824 raft_consensus.cc:2749] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:18.232534  2741 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:18.232698  2844 raft_consensus.cc:2749] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:18.244210  2850 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:18.250456 27901 tablet_replica.cc:333] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.251425 27901 raft_consensus.cc:2243] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.252794 27901 raft_consensus.cc:2272] T bf1e3918423546bcafc9359aca2ca811 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.259469 28976 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.262475 28976 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.263402 28976 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.265355 27901 tablet_replica.cc:333] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.266395 27901 raft_consensus.cc:2243] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.262063  2850 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:18.260447  2824 raft_consensus.cc:2749] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:18.268319  2844 raft_consensus.cc:2749] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:18.270426 28976 leader_election.cc:304] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.271478 28976 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.272337 28976 leader_election.cc:304] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.273286 28976 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.269451  2823 raft_consensus.cc:2749] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:18.274395  2850 raft_consensus.cc:2749] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:18.276684  2844 raft_consensus.cc:2749] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:18.279567 28976 raft_consensus.cc:2686] T 2bf52cc065f943c9a83088bbeea5c794 P 06ae4197fd784525b695a9b16c8094ea: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:13:18.286446 28976 leader_election.cc:304] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.278290  2741 raft_consensus.cc:2749] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:18.287640 27901 raft_consensus.cc:2272] T 49241be91ca743cbac56eb24218ff775 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.291430 28976 leader_election.cc:304] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.294978  2867 raft_consensus.cc:2749] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:18.291337  2824 raft_consensus.cc:2749] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:18.302130 28976 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:18.304212  2867 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:18.315096 28976 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06ae4197fd784525b695a9b16c8094ea; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.317655 27901 tablet_replica.cc:333] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
W20260709 13:13:18.322492 28976 raft_consensus.cc:2686] T dafcee70c76341f2a36a461c97f2b3ba P 06ae4197fd784525b695a9b16c8094ea: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:13:18.324512 27901 raft_consensus.cc:2243] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.333937 27901 raft_consensus.cc:2272] T 00c2e050cffc43618bffbaaa661fcfa6 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.355726 29713 raft_consensus.cc:1217] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef [term 11 LEARNER]: Deduplicated request from leader. Original: 11.9->[11.10-11.10]   Dedup: 11.10->[]
I20260709 13:13:18.380970  2832 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:18.381980  2832 tablet_bootstrap.cc:492] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:13:18.382632  2832 ts_tablet_manager.cc:1403] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.261s	user 0.039s	sys 0.008s
I20260709 13:13:18.409061 27901 tablet_replica.cc:333] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.424978 27901 raft_consensus.cc:2243] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.425693 27901 raft_consensus.cc:2272] T 2bbf1db4b075436695fd521f5a5b2e9d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.427416  2832 raft_consensus.cc:359] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:18.428395  2832 raft_consensus.cc:740] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:13:18.429091  2832 consensus_queue.cc:260] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:18.452606 27901 tablet_replica.cc:333] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.469857 27901 raft_consensus.cc:2243] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.470883 27901 raft_consensus.cc:2272] T 26c8dceb42c541c8906c424b9bc9bb17 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.472559  2832 ts_tablet_manager.cc:1434] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.001s
I20260709 13:13:18.482949 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-2bbf1db4b075436695fd521f5a5b2e9d received from {username='slave'} at 127.0.0.1:39900
I20260709 13:13:18.483371 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-2bbf1db4b075436695fd521f5a5b2e9d on tablet 2bbf1db4b075436695fd521f5a5b2e9d with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.498164  2825 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:18.499169  2825 tablet_bootstrap.cc:492] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
I20260709 13:13:18.499826  2825 ts_tablet_manager.cc:1403] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.465s	user 0.035s	sys 0.023s
I20260709 13:13:18.501819 27901 tablet_replica.cc:333] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.502882 27901 raft_consensus.cc:2243] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.503468 27901 raft_consensus.cc:2272] T 640d00aba1b541e4abb90c63e6b8720d P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.516109 27901 tablet_replica.cc:333] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.516901 27901 raft_consensus.cc:2243] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.517477 27901 raft_consensus.cc:2272] T 1db879e214974ab38bc94a40afcf6e1e P 06ae4197fd784525b695a9b16c8094ea [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.520126 27901 tablet_replica.cc:333] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.520912 27901 raft_consensus.cc:2243] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.521477 27901 raft_consensus.cc:2272] T 8da4f9ce63d54a63a8b9de11ba485699 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.534986  2825 raft_consensus.cc:359] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:18.535952  2825 raft_consensus.cc:740] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:13:18.536646  2825 consensus_queue.cc:260] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:18.539057  2825 ts_tablet_manager.cc:1434] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:13:18.540901 29169 tablet_copy_service.cc:342] P 243d4ae4a4ae48b2979e32874472d256: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-d012d02d5bf34cbf8f4451c054b36f63 received from {username='slave'} at 127.0.0.1:32836
I20260709 13:13:18.541317 29169 tablet_copy_service.cc:434] P 243d4ae4a4ae48b2979e32874472d256: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-d012d02d5bf34cbf8f4451c054b36f63 on tablet d012d02d5bf34cbf8f4451c054b36f63 with peer ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:18.523984 27901 tablet_replica.cc:333] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.542814 27901 raft_consensus.cc:2243] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.543423 27901 raft_consensus.cc:2272] T 342b8b8f6e7742c897bcd3495fecb00a P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.546144 27901 tablet_replica.cc:333] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.546970 27901 raft_consensus.cc:2243] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.552723 27901 raft_consensus.cc:2272] T d83f8e6d06054712b0c7cd2c5690ec63 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.579316 27901 tablet_replica.cc:333] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.580155 27901 raft_consensus.cc:2243] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.581224 27901 raft_consensus.cc:2272] T 3eaf29b1a5c049b1bc9159a53ce52afb P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.596608 27901 tablet_replica.cc:333] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.597453 27901 raft_consensus.cc:2243] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.598520 27901 raft_consensus.cc:2272] T 72d50ab8517242a1bb42501ec563c433 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.625922 27901 tablet_replica.cc:333] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.626752 27901 raft_consensus.cc:2243] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.627861 27901 raft_consensus.cc:2272] T 44c8fda62bd34c2f87eb2114b065741e P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.630475 27901 tablet_replica.cc:333] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.631246 27901 raft_consensus.cc:2243] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.632236 27901 raft_consensus.cc:2272] T f68cf8a7e5894c679a8bba8d7d2374a5 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.640576 27901 tablet_replica.cc:333] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.645678 27901 raft_consensus.cc:2243] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.646644 27901 raft_consensus.cc:2272] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.649307 27901 tablet_replica.cc:333] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.650051 27901 raft_consensus.cc:2243] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.650561 27901 raft_consensus.cc:2272] T 3aa255ba50b343eca7717515ec37d959 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.659126 27901 tablet_replica.cc:333] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.659870 27901 raft_consensus.cc:2243] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.663511 27901 raft_consensus.cc:2272] T e21ca94fafcc4923a671c355b710f10d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.671306 27901 tablet_replica.cc:333] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.672016 27901 raft_consensus.cc:2243] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.672605 27901 raft_consensus.cc:2272] T 7fb3976ea75a4cb3b4f543d842c2409c P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.674994 27901 tablet_replica.cc:333] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.675765 27901 raft_consensus.cc:2243] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.676786 27901 raft_consensus.cc:2272] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.686812 27901 tablet_replica.cc:333] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.687726 27901 raft_consensus.cc:2243] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.688347 27901 raft_consensus.cc:2272] T 57dd7e7308fc43e0b0983412129aa471 P 06ae4197fd784525b695a9b16c8094ea [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.691074 27901 tablet_replica.cc:333] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.691715 27901 raft_consensus.cc:2243] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.692721 27901 raft_consensus.cc:2272] T 7216d7dc49494630845e9eeb30dc6902 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.702100 27901 tablet_replica.cc:333] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.702951 27901 raft_consensus.cc:2243] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.703521 27901 raft_consensus.cc:2272] T 5cac0f90a42148b5923aa84c7b7cf00b P 06ae4197fd784525b695a9b16c8094ea [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.717527 27901 tablet_replica.cc:333] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.718412 27901 raft_consensus.cc:2243] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.719091 27901 raft_consensus.cc:2272] T e583be49bed64dd2996afcdcffa95120 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.721801 27901 tablet_replica.cc:333] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.722551 27901 raft_consensus.cc:2243] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.733028 27901 raft_consensus.cc:2272] T 828bc1c562f941c89bdcc43cf79e5ece P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.735921 27901 tablet_replica.cc:333] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.736698 27901 raft_consensus.cc:2243] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.737303 27901 raft_consensus.cc:2272] T 86e87ad9e9a5439d890ebaf60f043eb6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.741546 29713 raft_consensus.cc:1217] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 LEARNER]: Deduplicated request from leader. Original: 16.11->[16.12-16.12]   Dedup: 16.12->[]
I20260709 13:13:18.773353 27901 tablet_replica.cc:333] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.784404 27901 raft_consensus.cc:2243] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.785197 27901 raft_consensus.cc:2272] T 2185927c34434da38aa9e7776e483b1b P 06ae4197fd784525b695a9b16c8094ea [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.789106  2859 raft_consensus.cc:1064] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:13:18.794328 27901 tablet_replica.cc:333] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.791657  2859 consensus_queue.cc:237] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:18.795313 27901 raft_consensus.cc:2243] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.796644 27901 raft_consensus.cc:2272] T e225d0308db94fb9ab850a0b06a63551 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.799670 27901 tablet_replica.cc:333] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.800578 27901 raft_consensus.cc:2243] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.801435 27901 raft_consensus.cc:2272] T d012d02d5bf34cbf8f4451c054b36f63 P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.816764 29713 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef [term 11 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:18.811071 27901 tablet_replica.cc:333] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.818331 27901 raft_consensus.cc:2243] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.820339  2848 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:18.827175 29083 raft_consensus.cc:1275] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:18.827342 27901 raft_consensus.cc:2272] T 99a1616611dd41e49ec3ffc7eb3136a9 P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.831670  2848 consensus_queue.cc:1048] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:13:18.842327 27901 tablet_replica.cc:333] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.843286 27901 raft_consensus.cc:2243] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 13 LEADER]: Raft consensus shutting down.
W20260709 13:13:18.846212 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:18.852782 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:18.855820 27901 raft_consensus.cc:2272] T ab75774d33164f0b9c0269be22426768 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.866159 27901 tablet_replica.cc:333] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.866906 27901 raft_consensus.cc:2243] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.867501 27901 raft_consensus.cc:2272] T 21d163c72e704bd0bd37c74d96761663 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.876482 27901 tablet_replica.cc:333] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.877204 27901 raft_consensus.cc:2243] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:13:18.878149 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 48122.5us
I20260709 13:13:18.885720 27901 raft_consensus.cc:2272] T e1a1cae5d4eb46b288c03a3675d24209 P 06ae4197fd784525b695a9b16c8094ea [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:18.894271 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:18.895061 27901 tablet_replica.cc:333] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.896445 27901 raft_consensus.cc:2243] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.897078 27901 raft_consensus.cc:2272] T d8340da85294490fb88f62f80393531f P 06ae4197fd784525b695a9b16c8094ea [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.899981 27901 tablet_replica.cc:333] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.908854 27901 raft_consensus.cc:2243] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.910633 27901 raft_consensus.cc:2272] T ea8e44f6f34a4291baf426298bc260e6 P 06ae4197fd784525b695a9b16c8094ea [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.942250 27901 tablet_replica.cc:333] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.942929 27901 raft_consensus.cc:2243] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:18.943594 27901 raft_consensus.cc:2272] T 3dcde0796c504aa0ad4a5e79df6a169d P 06ae4197fd784525b695a9b16c8094ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.958951  2884 raft_consensus.cc:2955] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Committing config change with OpId 11.11: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:18.961959 29084 raft_consensus.cc:2955] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:13:18.978948 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:18.979470 27901 tablet_replica.cc:333] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea: stopping tablet replica
I20260709 13:13:18.980273 27901 raft_consensus.cc:2243] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:18.981357 27901 raft_consensus.cc:2272] T ddba9f7f2a6843c480dac4ac4efa4931 P 06ae4197fd784525b695a9b16c8094ea [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:18.995345 29713 raft_consensus.cc:2955] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:18.996196 28899 catalog_manager.cc:5697] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: config changed from index 10 to 11, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:19.033603 29713 raft_consensus.cc:1217] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef [term 15 LEARNER]: Deduplicated request from leader. Original: 15.7->[15.8-15.8]   Dedup: 15.8->[]
W20260709 13:13:19.088297 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:19.096112 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:19.131753 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.132508  2884 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.133090  2884 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:19.135486  2884 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:19.136477 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:19.140666  2859 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.141258  2859 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:19.144953  2886 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.145126 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:19.145565  2886 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:19.148475  2886 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:19.143864  2859 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:19.149721 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:13:19.154331 29122 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:13:19.154846 29122 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:19.156258  2859 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:19.183008 29122 leader_election.cc:336] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:13:19.183591 29122 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:19.192247  2859 raft_consensus.cc:2749] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:13:19.193761 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:19.194370 29122 leader_election.cc:336] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:13:19.195125 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:19.197141  2859 raft_consensus.cc:2749] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:19.214393  2834 raft_consensus.cc:1064] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
W20260709 13:13:19.217679 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.216887  2834 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:19.216733 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:19.233294 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:19.242276 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.242731 29714 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:19.244338 29156 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:19.246047  2876 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:19.247397  2834 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:13:19.257243 29048 consensus_peers.cc:597] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:19.306964  2876 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.307539  2876 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:19.311935  2834 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 12 to 13, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:19.319370 29714 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:13:19.326397 29048 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:13:19.327818  2876 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:19.328078 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:19.328832 29156 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:19.330061 29050 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:19.331048  2876 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:19.331409  2876 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.331737  2876 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:19.344889 29156 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:19.370417 28906 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 12 to 13, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } attrs { replace: true } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:19.394804  2834 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.396494  2876 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:19.401608  2838 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.413066  2838 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:19.414461  2885 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.415134  2885 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:19.416208  2838 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:19.395387  2834 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:19.420871  2885 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:19.424379 29680 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:13:19.432236 29680 leader_election.cc:336] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:13:19.436379  2886 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:13:19.437206 29048 leader_election.cc:336] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:13:19.436954  2886 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:19.437364 29084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:19.437973 29083 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:19.439035 29683 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:19.440270  2838 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:19.441812  2886 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:19.443051  2834 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:19.443540  2876 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:19.446458 29048 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:13:19.453229 29122 leader_election.cc:336] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:13:19.453435 29714 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:19.455327  2876 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:19.465812  2863 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:19.476636 29714 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:19.487396 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:19.487645 28895 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2bbf1db4b075436695fd521f5a5b2e9d with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:13:19.488236 29156 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 18.
I20260709 13:13:19.490907 29682 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:19.492237  2838 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:19.492682  2838 raft_consensus.cc:493] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.493108  2838 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:19.499658 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.505520  2863 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.506142  2863 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:19.507422  2838 raft_consensus.cc:515] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:19.508917 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:19.510444 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:19.511018 29156 raft_consensus.cc:3060] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:19.509582 29150 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 18.
I20260709 13:13:19.519727 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:19.521224 29159 raft_consensus.cc:1275] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:19.524624  2834 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:19.525137  2834 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.525540  2834 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:19.525576  2863 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:19.527275 28905 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 13 to 14, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New cstate: current_term: 16 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:19.534727 29713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:19.538295  2846 consensus_queue.cc:1048] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:13:19.539160 29048 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:13:19.542121 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.549222 29156 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
I20260709 13:13:19.546888 29713 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
W20260709 13:13:19.557000 28973 connection.cc:466] server connection from 127.0.0.1:34936 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34936 (request call id 3682) could send its response
I20260709 13:13:19.557678  2838 leader_election.cc:290] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:19.558924 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a08e186ea0749ae979329103591a933" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:19.559482 29156 raft_consensus.cc:3060] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:19.560768 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: recv got EOF from 127.27.63.65:43941 (error 108). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.559504 29051 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:19.563411  2876 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:19.563839  2876 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.564247  2876 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:19.571481 29050 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:19.574533 29156 raft_consensus.cc:2468] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 19.
I20260709 13:13:19.578958 29682 leader_election.cc:304] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:19.576079  2863 raft_consensus.cc:2804] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:19.580897  2838 raft_consensus.cc:2804] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:19.581374  2838 raft_consensus.cc:697] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 19 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:13:19.582175  2838 consensus_queue.cc:237] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:19.596001  2899 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.598134  2834 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:19.596628  2899 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:19.600620  2834 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:19.586467  2876 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:19.616544  2876 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:19.616405 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec27c4bcd6b4eb3a5165f616c4ad641" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:19.617578 29150 raft_consensus.cc:3060] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:19.622382 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 196675us
I20260709 13:13:19.629210 29156 raft_consensus.cc:2955] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:19.636656 29150 raft_consensus.cc:2468] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 19.
I20260709 13:13:19.643066 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:19.660377 29050 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
W20260709 13:13:19.676251 29680 leader_election.cc:336] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:19.676522  2834 raft_consensus.cc:2804] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:19.691176 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824fc688db264856b025a29c651eec78" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:19.692116 29714 raft_consensus.cc:3060] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:19.678642  2863 raft_consensus.cc:697] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:19.706723 29714 raft_consensus.cc:2468] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:19.688244  2884 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.708463  2884 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:19.710213 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.710709  2884 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:13:19.746627 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 44790.9us
I20260709 13:13:19.756778 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:19.748216 28899 catalog_manager.cc:5697] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 18 to 19, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 19 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:19.793591 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 105835us
W20260709 13:13:19.808485 28892 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 122261us
I20260709 13:13:19.752115  2834 raft_consensus.cc:697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 19 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:19.819238 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:19.819869 29083 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:13:19.754070  2899 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:19.840798 29051 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:19.842366 29125 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:19.876549 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 72039.5us
I20260709 13:13:19.881639  2915 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.882220  2915 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:19.884250  2915 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:19.885990  2915 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.886400  2915 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:19.888000  2915 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:19.863734  2899 raft_consensus.cc:2804] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:13:19.889109 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 46807.1us
I20260709 13:13:19.898762  2898 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.899379  2898 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:19.888789  2859 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.905592  2859 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:19.906855 28892 catalog_manager.cc:4755] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2bbf1db4b075436695fd521f5a5b2e9d: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:19.907603  2859 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:19.878479  2834 consensus_queue.cc:237] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:19.910060 29048 leader_election.cc:336] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:19.919529 29048 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:19.920926 29048 leader_election.cc:336] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:19.921418 29048 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:13:19.948173 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 68941.6us
I20260709 13:13:19.892035  2884 raft_consensus.cc:2749] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:19.949544 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.844259  2863 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:13:19.956022 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:19.919077  2899 raft_consensus.cc:697] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:19.946660  2904 raft_consensus.cc:2749] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:19.955536 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:19.956367  2918 raft_consensus.cc:1064] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:13:19.950666  2916 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:13:19.980363  2812 ts_tablet_manager.cc:1011] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Tablet Copy: Unable to fetch data from remote peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Unable to download WAL segment with seq. number 1: unable to fetch data from remote: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:19.992931  2812 tablet_replica.cc:333] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:19.993271  2812 raft_consensus.cc:2243] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 16 LEARNER]: Raft consensus shutting down.
I20260709 13:13:19.993638  2812 raft_consensus.cc:2272] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 16 LEARNER]: Raft consensus is shut down!
I20260709 13:13:19.980185 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:19.983649 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:19.995924 29156 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:19.987865  2884 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:19.996029 29150 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:19.957497  2912 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:19.997877 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:13:20.000751 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:20.065938 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 60167.2us
I20260709 13:13:20.059036  2886 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.043987  2899 consensus_queue.cc:237] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.058573  2908 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.074486 29083 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 18.
I20260709 13:13:19.995348  2909 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:13:20.080698 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:20.084875 29048 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.085364 29048 leader_election.cc:336] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:19.999404  2884 raft_consensus.cc:493] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:13:20.082219 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:20.087348 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:20.062691  2914 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:13:20.087823 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:20.090153 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:13:20.084059  2812 tablet_copy_client.cc:1131] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.094054  2812 tablet_metadata.cc:476] failed to load DataDirGroup from superblock: Already present: tried to load directory group for tablet bf1e3918423546bcafc9359aca2ca811 but one is already registered
I20260709 13:13:20.094410  2812 ts_tablet_manager.cc:1916] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:13:20.080850 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 80171.5us
I20260709 13:13:20.095774 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.083633  2921 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:13:20.095616 29122 leader_election.cc:336] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.095280  2914 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.089594  2909 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.103508  2834 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:20.103911  2834 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.104733  2834 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.085784  2876 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.087990  2884 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.106719  2921 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.113029  2921 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:19.998984  2916 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:20.125032 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:20.125504 29122 leader_election.cc:336] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.130676  2916 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.109902  2876 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.133116  2876 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.108570  2846 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:20.134867  2846 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.135247  2846 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:20.128083 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:20.141414  2914 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:20.144166  2909 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:20.087224  2908 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.152609  2908 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:20.086318  2886 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.154690  2886 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:20.168506  2898 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:20.170281 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:20.181465  2898 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.182111  2898 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:20.184661 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:20.185082  2898 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.058660  2912 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:20.190902 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 98097.6us
I20260709 13:13:20.194589  2846 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.195274  2834 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:20.197909 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:20.201472 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:20.202105 29150 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 18.
I20260709 13:13:20.207535 29158 raft_consensus.cc:1275] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:13:20.208614 29714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:20.209215 29714 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 18.
W20260709 13:13:20.210323 29048 leader_election.cc:336] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.210848 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:20.211248 29048 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.211717 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.212808 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:20.213565  2916 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:20.213932  2916 raft_consensus.cc:493] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.214294  2916 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:20.215785 29077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:20.216179 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:20.216415 29077 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 14.
I20260709 13:13:20.216742 29082 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:13:20.218034 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:20.218808 29079 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:13:20.219996 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:20.220091 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:20.220646 29152 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
I20260709 13:13:20.220856 29156 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 18.
I20260709 13:13:20.221550 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:20.222196 29155 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:13:20.222604  2846 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.181734  2884 raft_consensus.cc:515] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:20.226537 29048 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.227648  2930 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.228207  2930 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.230214 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00c2e050cffc43618bffbaaa661fcfa6" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:20.230839 29158 raft_consensus.cc:3060] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.230926 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:20.231586 29085 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 13.
I20260709 13:13:20.236421  2912 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.238384  2884 leader_election.cc:290] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:20.230338  2930 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:13:20.245692 29048 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.247761 29680 leader_election.cc:336] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.248277  2897 consensus_queue.cc:1048] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:13:20.247263  2918 consensus_queue.cc:237] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 6, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.251879 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.253944 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.254967 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.255625  2929 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:20.256098  2929 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.256281  2911 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:20.256467  2929 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.256742  2911 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.257186  2911 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:20.258040  2876 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:20.258437  2876 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.258769  2876 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:20.261538 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:20.262169 29156 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 13.
I20260709 13:13:20.264081 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.265136  2846 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:20.265579  2846 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.266029  2846 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:20.266254 28899 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 16 to 17, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.268303  2916 raft_consensus.cc:515] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.270414  2916 leader_election.cc:290] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.285249 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c8dceb42c541c8906c424b9bc9bb17" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:20.271025  2911 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.282815  2834 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.288655 29150 raft_consensus.cc:3060] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.289024 29158 raft_consensus.cc:2468] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
I20260709 13:13:20.290050  2911 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:20.291121 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.271610 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb070bde5f3744eb90ee5cf3dec8fcd4" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:13:20.273442 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 147513us
I20260709 13:13:20.293788 29156 raft_consensus.cc:3060] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:20.293828 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:20.294570 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:20.302624 29085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d50ab8517242a1bb42501ec563c433" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:20.303217 29085 raft_consensus.cc:3060] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.305369 29150 raft_consensus.cc:2468] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 17.
I20260709 13:13:20.302865 29050 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.310446 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eaf29b1a5c049b1bc9159a53ce52afb" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:20.311074 29158 raft_consensus.cc:3060] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:20.313671 29050 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:20.310441 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:20.316347  2911 raft_consensus.cc:2804] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:20.317071  2911 raft_consensus.cc:697] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:20.318074  2904 raft_consensus.cc:2804] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:20.317905  2911 consensus_queue.cc:237] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.318567  2904 raft_consensus.cc:697] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:20.319676  2904 consensus_queue.cc:237] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.325299 29085 raft_consensus.cc:2468] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:13:20.325773  2846 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.328197  2846 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.330173 29156 raft_consensus.cc:2468] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 19.
I20260709 13:13:20.334587 29682 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:20.336150 29158 raft_consensus.cc:2468] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 19.
I20260709 13:13:20.332790 29125 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.337970  2927 raft_consensus.cc:2804] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:20.339684 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cf8a7e5894c679a8bba8d7d2374a5" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:20.344745 29156 raft_consensus.cc:3060] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:20.348320  2916 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.348923  2896 raft_consensus.cc:2764] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:13:20.349259  2927 raft_consensus.cc:697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 19 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
W20260709 13:13:20.350190 29122 leader_election.cc:336] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.350283  2927 consensus_queue.cc:237] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.348829  2916 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.352458 29050 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:20.353725  2912 raft_consensus.cc:2804] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:20.354296  2912 raft_consensus.cc:697] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 19 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:20.354595  2916 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.355146  2912 consensus_queue.cc:237] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:20.358501 29122 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.359346 29122 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.360203 29122 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.330978 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:20.360771 29122 leader_election.cc:336] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.361418 29122 leader_election.cc:336] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.363161 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:20.362534 29125 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.363848 29158 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
W20260709 13:13:20.361883 29122 leader_election.cc:336] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.367672 29048 leader_election.cc:336] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.373356 29048 leader_election.cc:336] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.375329 29125 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.376595  2895 raft_consensus.cc:1027] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: attempt to promote peer ca45420e033847489ad989c1c6eed3ef: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:13:20.379572  2923 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:20.372417 29682 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:20.382848 29156 raft_consensus.cc:2468] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 14.
I20260709 13:13:20.384861 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.383935 29050 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 
I20260709 13:13:20.353772  2812 ts_tablet_manager.cc:1929] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.0
I20260709 13:13:20.393870  2898 raft_consensus.cc:2804] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:20.393951  2897 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.394490  2898 raft_consensus.cc:697] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:20.394549  2897 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.395437  2898 consensus_queue.cc:237] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.389676  2916 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:20.396469  2916 raft_consensus.cc:493] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.396847  2916 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:20.397801  2897 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.400523 29125 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.391630  2812 log.cc:1199] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Deleting WAL directory at /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602591602838-27901-0/minicluster-data/ts-3-root/wals/bf1e3918423546bcafc9359aca2ca811
I20260709 13:13:20.387738  2876 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.417809  2886 raft_consensus.cc:2764] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:13:20.401995 29048 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.422731  2876 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.418229  2895 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:20.424134  2895 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.424551  2895 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.389919  2930 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:20.401798 29125 leader_election.cc:304] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:13:20.420009 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.379936  2929 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.388271  2923 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.427990 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:20.429034 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:20.433000 29125 leader_election.cc:304] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.431787 29156 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:13:20.430877  2930 raft_consensus.cc:493] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
W20260709 13:13:20.437072 29680 leader_election.cc:336] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.438545 29125 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:20.437095  2930 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.443886  2859 raft_consensus.cc:2764] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:13:20.445693  2929 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.445034 28899 catalog_manager.cc:5697] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 18 to 19, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 19 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.432075  2923 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:20.446944 28899 catalog_manager.cc:5697] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 15 to 16, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 16 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.441026 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:20.453681  2897 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:20.454064  2897 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election (detected failure of leader 06ae4197fd784525b695a9b16c8094ea)
I20260709 13:13:20.451857  2884 raft_consensus.cc:2749] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:20.454401  2897 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:20.454525 29716 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef [term 15 LEARNER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:20.457521  2916 raft_consensus.cc:515] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.458415 28905 catalog_manager.cc:5697] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 18 to 19, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 19 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.460094  2930 raft_consensus.cc:515] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.461228 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab75774d33164f0b9c0269be22426768" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:20.461778 29152 raft_consensus.cc:3060] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:20.466058  2896 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:20.469015  2930 leader_election.cc:290] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:20.437646  2908 raft_consensus.cc:2804] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:20.469874  2908 raft_consensus.cc:697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:20.470736  2908 consensus_queue.cc:237] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.478118 29152 raft_consensus.cc:2468] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:13:20.481389 29682 leader_election.cc:304] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:20.482375  2917 raft_consensus.cc:2804] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:20.482889  2917 raft_consensus.cc:697] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:13:20.483757  2917 consensus_queue.cc:237] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.485419 29082 raft_consensus.cc:1275] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:20.489988 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:20.491632 29082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a1616611dd41e49ec3ffc7eb3136a9" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:20.492247 29082 raft_consensus.cc:3060] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.487870  2916 leader_election.cc:290] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:20.517020 29680 leader_election.cc:336] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:20.518344 29680 leader_election.cc:336] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.519331  2923 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.527108  2923 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:13:20.527917 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.528191 29158 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 15: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:13:20.528555 29048 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.536317 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:20.537195 29083 raft_consensus.cc:2393] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 15: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:13:20.515781  2859 consensus_queue.cc:1048] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:13:20.538563 29152 raft_consensus.cc:1275] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:13:20.551486 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:20.552796 29158 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:13:20.549984 29122 leader_election.cc:336] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.559643 29082 raft_consensus.cc:2468] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:13:20.564937 29125 leader_election.cc:304] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:20.559773  2876 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:20.564266  2911 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:13:20.575102  2923 raft_consensus.cc:2804] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:20.583638  2923 raft_consensus.cc:697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:20.584808  2923 consensus_queue.cc:237] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.585305  2895 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:20.586623 29156 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 16: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
W20260709 13:13:20.588830 27901 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:20.579182 29122 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.589556 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:20.591454 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
W20260709 13:13:20.601750 29051 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167): Couldn't send request to peer ca45420e033847489ad989c1c6eed3ef. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:20.603096 29680 leader_election.cc:336] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.605015  2859 raft_consensus.cc:2749] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:20.559980  2904 consensus_queue.cc:1048] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:13:20.649462  2895 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:13:20.650115 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.616271  2846 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:20.664137 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:20.669369 28899 catalog_manager.cc:5697] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 16 to 17, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.671667 28899 catalog_manager.cc:5697] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 18 to 19, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 19 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.673405 28899 catalog_manager.cc:5697] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 16 to 17, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.675282 28899 catalog_manager.cc:5697] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 13 to 14, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 14 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.680250 29083 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 16: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
W20260709 13:13:20.696779 29122 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.704703 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:20.717325  2897 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:20.719426  2897 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:20.725260  2895 raft_consensus.cc:2955] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Committing config change with OpId 15.9: config changed from index 8 to 9, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:20.713887  2923 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:20.737087 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:20.737727 29150 raft_consensus.cc:3060] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:20.755729 28906 catalog_manager.cc:5697] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 13 to 14, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 14 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.748337 28905 catalog_manager.cc:5697] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 16 to 17, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 17 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:20.764082 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:20.764787 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.768126 29082 raft_consensus.cc:2955] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index 8 to 9, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:20.779495 29150 raft_consensus.cc:2468] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 15.
W20260709 13:13:20.791003 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.790599 29717 raft_consensus.cc:2955] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index 8 to 9, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:20.800951 29682 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:20.805445  2917 raft_consensus.cc:2804] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:20.806057  2917 raft_consensus.cc:697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 15 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:13:20.806954  2917 consensus_queue.cc:237] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:20.818583 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.827036 29716 raft_consensus.cc:1275] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:20.839701 29150 raft_consensus.cc:1275] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:20.846536  2904 consensus_queue.cc:1048] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:20.860527  2898 consensus_queue.cc:1048] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:13:20.867316 29680 leader_election.cc:336] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:20.969552 28905 catalog_manager.cc:5697] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 16 to 17, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 17 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.974083 29150 raft_consensus.cc:1275] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:13:20.976202  2906 consensus_queue.cc:1048] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:13:20.980717 28905 catalog_manager.cc:5697] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: config changed from index 8 to 9, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:21.015525 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:21.033221 29150 raft_consensus.cc:1275] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:13:21.044921 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:21.045576 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:21.053499 29150 raft_consensus.cc:1275] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:21.058112 29152 raft_consensus.cc:1275] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:13:21.063200 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:21.066102  2958 ts_tablet_manager.cc:933] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Initiating tablet copy from peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:21.087831 29083 raft_consensus.cc:1275] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:21.100490 28906 catalog_manager.cc:5697] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 14 to 15, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 15 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:21.108986 29150 raft_consensus.cc:1275] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:21.112903 29082 raft_consensus.cc:1275] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:13:21.133530  2923 consensus_queue.cc:1048] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:13:21.133625  2946 consensus_queue.cc:1048] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:13:21.146415  2958 tablet_copy_client.cc:323] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Beginning tablet copy session from remote peer at address 127.27.63.66:35939
W20260709 13:13:21.128085 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:21.154956 29095 tablet_copy_service.cc:140] P befda4a6c8344c26b0f139e4e1686b3b: Received BeginTabletCopySession request for tablet bf1e3918423546bcafc9359aca2ca811 from peer ca45420e033847489ad989c1c6eed3ef ({username='slave'} at 127.0.0.1:39900)
I20260709 13:13:21.155684 29095 tablet_copy_service.cc:161] P befda4a6c8344c26b0f139e4e1686b3b: Beginning new tablet copy session on tablet bf1e3918423546bcafc9359aca2ca811 from peer ca45420e033847489ad989c1c6eed3ef at {username='slave'} at 127.0.0.1:39900: session id = ca45420e033847489ad989c1c6eed3ef-bf1e3918423546bcafc9359aca2ca811
I20260709 13:13:21.156946  2911 consensus_queue.cc:1048] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:21.155385  2925 consensus_queue.cc:1048] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:13:21.164295  2925 consensus_queue.cc:1048] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:21.179133  2921 consensus_queue.cc:1048] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:21.198150 29095 tablet_copy_source_session.cc:215] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:13:21.210088 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:21.222527 29156 raft_consensus.cc:1275] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Refusing update from remote peer ca45420e033847489ad989c1c6eed3ef: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:21.230050  2885 consensus_queue.cc:1048] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:21.264593  2958 ts_tablet_manager.cc:1916] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Deleting tablet data with delete state TABLET_DATA_COPYING
W20260709 13:13:21.335435 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:21.390419 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:21.394513 29048 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111) [suppressed 356 similar messages]
W20260709 13:13:21.417454 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51978.9us
I20260709 13:13:21.469616  2958 ts_tablet_manager.cc:1929] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.0
I20260709 13:13:21.470745  2958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf1e3918423546bcafc9359aca2ca811. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:21.520936 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:21.524102 28892 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 71451.9us
I20260709 13:13:21.543383  2958 tablet_copy_client.cc:806] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:21.544092  2958 tablet_copy_client.cc:670] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Starting download of 1 WAL segments...
W20260709 13:13:21.544521 28892 catalog_manager.cc:4755] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2bbf1db4b075436695fd521f5a5b2e9d: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:21.581120 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:21.583698  2958 tablet_copy_client.cc:538] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:13:21.593811 27901 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 29031)
I20260709 13:13:21.599928  2958 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Bootstrap starting.
I20260709 13:13:21.746964  2958 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:13:21.750025 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:21.758792  2958 tablet_bootstrap.cc:492] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Bootstrap complete.
W20260709 13:13:21.765239 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:21.763377  2958 ts_tablet_manager.cc:1403] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Time spent bootstrapping tablet: real 0.164s	user 0.037s	sys 0.012s
I20260709 13:13:21.771829  2958 raft_consensus.cc:359] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
I20260709 13:13:21.777711  2958 raft_consensus.cc:740] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Initialized, Role: LEARNER
I20260709 13:13:21.778445  2958 consensus_queue.cc:260] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: NON_VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: true } }
W20260709 13:13:21.779623 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:21.807742  2958 ts_tablet_manager.cc:1434] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.004s
I20260709 13:13:21.809609 29095 tablet_copy_service.cc:342] P befda4a6c8344c26b0f139e4e1686b3b: Request end of tablet copy session ca45420e033847489ad989c1c6eed3ef-bf1e3918423546bcafc9359aca2ca811 received from {username='slave'} at 127.0.0.1:39900
I20260709 13:13:21.810081 29095 tablet_copy_service.cc:434] P befda4a6c8344c26b0f139e4e1686b3b: ending tablet copy session ca45420e033847489ad989c1c6eed3ef-bf1e3918423546bcafc9359aca2ca811 on tablet bf1e3918423546bcafc9359aca2ca811 with peer ca45420e033847489ad989c1c6eed3ef
W20260709 13:13:21.890409 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:21.937726 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:21.958606 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:21.970630 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:21.971792 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:21.978539 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.025171 29716 raft_consensus.cc:1217] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Deduplicated request from leader. Original: 16.10->[17.11-17.11]   Dedup: 17.11->[]
W20260709 13:13:22.040186 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.088866 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.146950 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 58129.4us
W20260709 13:13:22.162717 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.327569  2904 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.328171  2904 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:22.330329  2904 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:22.343583 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:22.344527 29152 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 15.
W20260709 13:13:22.363415 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:22.364173 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.365415 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:22.369794  2899 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:22.370340  2899 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.370795  2899 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:22.398517 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.406716 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.416361 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 51857.5us
W20260709 13:13:22.420737 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.427031 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.433360  2896 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.437327  2896 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:22.440364  2896 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:13:22.452972 29122 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:22.457090  2898 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.457731  2898 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:22.461886 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.459945  2898 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:22.478789 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.482198 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.510771 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.512979  2899 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:22.546262  2899 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:22.536841  2898 raft_consensus.cc:1064] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: attempting to promote NON_VOTER ca45420e033847489ad989c1c6eed3ef to VOTER
I20260709 13:13:22.557775  2898 consensus_queue.cc:237] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:22.527770 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 44667us
W20260709 13:13:22.551628 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.594307 27901 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 29031)
I20260709 13:13:22.603706 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:22.604550 29152 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
I20260709 13:13:22.606138 29156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7216d7dc49494630845e9eeb30dc6902" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:22.606703 29156 raft_consensus.cc:3060] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:22.615471 29156 raft_consensus.cc:2468] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate befda4a6c8344c26b0f139e4e1686b3b in term 16.
W20260709 13:13:22.623383 29048 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:22.641038 29079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:22.657351 29079 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
W20260709 13:13:22.658033 29048 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:22.658684 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:22.669737 29716 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 LEARNER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:13:22.671243  2898 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:22.675726 29050 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:22.676659  2973 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:22.677148  2973 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.677495  2973 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:22.679270 28892 catalog_manager.cc:4755] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2bbf1db4b075436695fd521f5a5b2e9d: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:22.687916  2898 raft_consensus.cc:2804] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:22.689126  2898 raft_consensus.cc:697] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Becoming Leader. State: Replica: befda4a6c8344c26b0f139e4e1686b3b, State: Running, Role: LEADER
I20260709 13:13:22.690572  2898 consensus_queue.cc:237] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:22.696604 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:22.697482 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:22.710536 29150 raft_consensus.cc:1275] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:13:22.715999 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:22.718969  2971 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:22.721071  2973 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:22.722097  2971 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.722500  2971 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:22.723722  2973 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:22.719061  2899 consensus_queue.cc:1048] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
W20260709 13:13:22.721632 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.728366 29152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "befda4a6c8344c26b0f139e4e1686b3b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
W20260709 13:13:22.729257 29048 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:22.739753 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.769476 28901 catalog_manager.cc:5697] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b reported cstate change: term changed from 14 to 16, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66). New cstate: current_term: 16 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:22.773002  2971 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:22.777431  2971 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:13:22.781625 29122 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:22.786954 29152 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate befda4a6c8344c26b0f139e4e1686b3b in current term 17: Already voted for candidate 243d4ae4a4ae48b2979e32874472d256 in this term.
I20260709 13:13:22.792769 29050 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:22.797019  2899 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:22.812947  2973 raft_consensus.cc:2955] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 10 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:13:22.815977 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.828660 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:22.829543 29083 raft_consensus.cc:2393] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 17: Already voted for candidate befda4a6c8344c26b0f139e4e1686b3b in this term.
I20260709 13:13:22.835322 29716 raft_consensus.cc:2955] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 10 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:13:22.851289 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:22.860913 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:22.862349  2971 raft_consensus.cc:2749] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:22.865190 29156 raft_consensus.cc:2955] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 10 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
I20260709 13:13:22.869935 28905 catalog_manager.cc:5697] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef reported cstate change: config changed from index 10 to 12, ca45420e033847489ad989c1c6eed3ef (127.27.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "befda4a6c8344c26b0f139e4e1686b3b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } } }
W20260709 13:13:22.872229 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:22.972502 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.036557 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.099117 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:23.144742 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:23.216711 29150 raft_consensus.cc:1275] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Refusing update from remote peer befda4a6c8344c26b0f139e4e1686b3b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:23.222767  2970 consensus_queue.cc:1048] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:13:23.223712 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.334530 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:23.339931 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.345786 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.407022 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.527128 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.528982 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.594794 27901 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:23.622430 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.670883 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 66953.2us
W20260709 13:13:23.739197 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:23.819569 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:23.981685 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:24.120287 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.124089 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.232502 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:24.282717 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.312577 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.333546 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.396416 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.399529 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:24.503453  2980 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.504146  2980 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:24.506991 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.520068 29122 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:24.528343 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:24.529179 29083 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:13:24.530498 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
I20260709 13:13:24.531468  2971 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:24.531916  2971 raft_consensus.cc:493] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:24.532433  2971 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:24.537328 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:24.556164  2980 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:13:24.595281 27901 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:24.600926 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.609009 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 56621.9us
I20260709 13:13:24.619254  2971 raft_consensus.cc:515] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:24.621618  2971 leader_election.cc:290] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 election: Requested vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:24.655253 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c8fda62bd34c2f87eb2114b065741e" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b"
I20260709 13:13:24.680421 29083 raft_consensus.cc:3060] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:24.685812 29122 leader_election.cc:336] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:24.670783 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.711565 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:24.733136 29083 raft_consensus.cc:2468] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 18.
W20260709 13:13:24.777704 28892 catalog_manager.cc:4755] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2bbf1db4b075436695fd521f5a5b2e9d: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:24.821254 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 67940.3us
W20260709 13:13:24.855744 28892 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 78065.2us
I20260709 13:13:24.873908 29125 leader_election.cc:304] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b; no voters: 06ae4197fd784525b695a9b16c8094ea
W20260709 13:13:24.884373 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:24.897773  2980 raft_consensus.cc:2804] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:24.898428  2980 raft_consensus.cc:697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 18 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:24.899518  2980 consensus_queue.cc:237] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
W20260709 13:13:24.907408 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:24.955663 28906 catalog_manager.cc:5697] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 18, leader changed from 06ae4197fd784525b695a9b16c8094ea (127.27.63.65) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 18 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:24.988526 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:24.999748 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 54660us
W20260709 13:13:25.004158 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 79566.5us
W20260709 13:13:25.007491 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.011750  2998 log.cc:927] Time spent T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
W20260709 13:13:25.021785 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.038192 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:25.069406 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.069957 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.076431 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.081223 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 74553us
W20260709 13:13:25.112224 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.113260 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.155349 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.184900 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:25.198297 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.198801 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.219105 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.333045 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.340794 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:25.367565 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:25.493086 29079 raft_consensus.cc:1275] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Refusing update from remote peer 243d4ae4a4ae48b2979e32874472d256: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:13:25.494536  2995 consensus_queue.cc:1048] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:13:25.503796 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:25.564400 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:25.565100 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:25.596194 27901 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:25.657428 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:25.681003 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:25.770576 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:25.853348 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:25.951071 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:25.963480 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:26.015167 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:26.143875 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:26.200593 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:26.201229 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:26.291934 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:26.413077 29680 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111) [suppressed 474 similar messages]
W20260709 13:13:26.442859 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:26.505986 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:26.524878 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 127980us
W20260709 13:13:26.597141 27901 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:26.667852 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:26.737746 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:26.793080 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 55374.3us
W20260709 13:13:26.871170 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:26.876825 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:26.933763 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:26.950748 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:27.039569 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.100524 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.101197 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.101639 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.103099 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.109453 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.187230 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.190218 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.209851 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.217500 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.245608 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.308284 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.309782 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 63512.3us
W20260709 13:13:27.406124 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.523039 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.556921 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.594730 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.597610 27901 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:27.609602 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:27.705485 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.730676 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.731254 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.743795 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.836311 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:27.877257 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.928328 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:27.965667 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:27.974900 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.003540 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 62857.1us
W20260709 13:13:28.038033 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.148841 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.205746 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 49933.9us
W20260709 13:13:28.252357 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46924.4us
W20260709 13:13:28.308828 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.321007 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.322794 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.325687 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:28.543074 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.568912 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 172334us
W20260709 13:13:28.598086 27901 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:28.652989 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.690086 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 51004.1us
W20260709 13:13:28.735337 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 54458.5us
W20260709 13:13:28.761345 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.768266 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:28.768787 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.767653 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:28.779969 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.780663 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:28.891131 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:28.990594 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:29.010456 28892 catalog_manager.cc:4755] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2bbf1db4b075436695fd521f5a5b2e9d: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:29.183251 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.201586 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.204734 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.272434 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.286160 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:29.389302 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.431011 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.560225 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.598284 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.598596 27901 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:29.620401 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.627612 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.648757 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.667833 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.720295 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.725687 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.751331 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.845647 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.891618 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.920315 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:29.990386 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.109751 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.170001 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.184386 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.198172 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:30.221818 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:30.241052 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.398077 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:30.428668 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.499191 29050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 119172us
W20260709 13:13:30.500326 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:30.501246 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.494216 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.592882 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:30.599023 27901 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:30.702977 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:30.704196 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:30.707231 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:30.885080 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:30.927378 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:30.944977 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:31.176743 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:31.182868 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:31.225113 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:31.227766 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 104871us
W20260709 13:13:31.229169 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 45832us
W20260709 13:13:31.244956 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:31.266966 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.291509 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.333773 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:31.416025 29122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111) [suppressed 471 similar messages]
W20260709 13:13:31.449625 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.478050 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.601379 27901 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:31.693660 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.744990 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.780787 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.799983 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:31.815778 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.878410 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.931800 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:31.941602 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:31.980343 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.023118 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.081017 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.159525 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.228979 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.264904 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.317605 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.348169 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.356587 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.428968 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.454010 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.480475 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.590303 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.593319 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.601858 27901 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:32.617858 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:32.668792 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.771790 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.799855 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:32.894550 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.898387 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:32.952600 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:32.995697 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.169768 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.215890 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.218832 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:33.248456 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.288343 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:33.383155 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.445988 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.601774 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.602321 27901 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:33.611413 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.642840 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:33.657896 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.720975 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.742276 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:33.750123 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:33.940348 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.028535 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.104988 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:34.165557 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.222158 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:34.242002 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.382887 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.446487 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.446969 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:34.450445 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.466625 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.551182 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.602758 27901 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:34.634253 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.686590 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.739416 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.767715 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.804291 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.816272 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.821213 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.864856 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.898202 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:34.915217 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:35.044708 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:35.060528 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:35.102790 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:35.139786 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:35.208127 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:35.222967 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:35.251340 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:35.312839 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:35.506204 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:35.735647 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:35.786352 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:35.887638 29682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:40957, user_credentials={real_user=slave}} blocked reactor thread for 87623.5us
W20260709 13:13:35.945410 27901 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:36.081275 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:36.108700 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 119688us
W20260709 13:13:36.168923 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.196915 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.299690 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.301420 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:36.302199 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:36.304613 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.306945 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.308074 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.356287 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:36.420598 29122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111) [suppressed 446 similar messages]
W20260709 13:13:36.474376 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:36.495893 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.518713 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:36.625711 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.626915 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:36.678726 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:36.707150 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.753862 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:36.824517 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:36.945960 27901 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:36.953691 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:36.964689 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:36.984267 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:37.005678 29106 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1f40 after lost signal to thread 27904
W20260709 13:13:37.006861 29106 debug-util.cc:398] Leaking SignalData structure 0x7b0800335620 after lost signal to thread 28942
W20260709 13:13:37.008214 29106 debug-util.cc:398] Leaking SignalData structure 0x7b0800147c80 after lost signal to thread 29109
W20260709 13:13:37.010937 29106 debug-util.cc:398] Leaking SignalData structure 0x7b08003c0e00 after lost signal to thread 29183
W20260709 13:13:37.014153 29106 debug-util.cc:398] Leaking SignalData structure 0x7b0800362420 after lost signal to thread 29741
W20260709 13:13:37.023809 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:37.066584 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.100485 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.132375 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.185736 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 53418.5us
W20260709 13:13:37.234140 29683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 56295.8us
W20260709 13:13:37.277026 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.277773 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.282099 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.306612 28892 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 50870.5us
W20260709 13:13:37.315362 28892 catalog_manager.cc:4755] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2bbf1db4b075436695fd521f5a5b2e9d: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:37.331707 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.342546 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.396130 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.414273 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.596195 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.650822 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.686831 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.729559 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.749291 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:37.875080 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.885054 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.934562 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:37.946473 27901 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:38.063393 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:38.246917 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:38.258392 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:38.271160 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.287683 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.306241 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.316846 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.348791 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.366580 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:38.382035 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:38.487303 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.633383 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:38.765697 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.791759 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.842919 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:38.865700 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:38.946981 27901 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:38.986677 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:38.994098 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.001012 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.008654 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:39.096458 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:39.124583 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:39.174388 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:39.178040 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.233861 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:39.321197 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.442988 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.450853 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:39.635573 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.648936 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.694123 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.701583 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.755787 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.790289 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.865149 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.868317 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.919716 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.920675 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:39.947487 27901 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:39.973219 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.086921 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.250751 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.259713 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.292847 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.393862 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.429659 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.451315 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:40.509609 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.542491 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.601342 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.638471 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:40.660338 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:40.694370 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:40.716861 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:40.728606 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:40.803013 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:40.851094 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:40.947958 27901 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:41.019728 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:41.030427 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:41.077875 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:41.210182 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:41.344600 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:41.391585 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:41.422744 29051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 119334us
W20260709 13:13:41.436295 29680 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111) [suppressed 472 similar messages]
W20260709 13:13:41.448119 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:41.456795 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:41.457515 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:41.466166 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:41.530021 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:41.646289 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:41.647890 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:41.669320 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:41.894496 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:41.897162 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:41.919191 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:41.948897 27901 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:42.057334 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:42.093637 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.105746 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.124437 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.282197 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.282804 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.297348 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.376092 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.390902 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.426790 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.502408 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.592948 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.634688 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.725827 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.821491 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.823777 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.829126 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.897248 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:42.949465 27901 thread.cc:527] Waited for 23000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:42.961154 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.967474 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:42.992112 29048 consensus_peers.cc:597] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:43.057910 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:43.060021 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:43.082473 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:43.241389 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:43.246155 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:43.289876 29048 consensus_peers.cc:597] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:43.358906 29048 consensus_peers.cc:597] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:43.400688 29048 consensus_peers.cc:597] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:43.530957 29048 consensus_peers.cc:597] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:43.551774 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:43.552426 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:43.649595 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:43.692315 29048 consensus_peers.cc:597] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:43.773164 29048 consensus_peers.cc:597] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:43.941213 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:43.951210 27901 thread.cc:527] Waited for 24000ms trying to join with diag-logger (tid 29031)
W20260709 13:13:43.974879 29048 consensus_peers.cc:597] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:43.978998 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:44.022459 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:44.063796 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:44.085846 29048 consensus_peers.cc:597] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:44.104390 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:44.136595 29048 consensus_peers.cc:597] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:44.304767 29048 consensus_peers.cc:597] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:44.390442 29048 consensus_peers.cc:597] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:44.407306 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.282437 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:45.287236 29122 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.303206 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.334707 29048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 43754.4us
I20260709 13:13:45.340018  3112 raft_consensus.cc:493] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:45.356477  3112 raft_consensus.cc:515] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:45.367717  3112 leader_election.cc:290] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
W20260709 13:13:45.392028 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:35939, user_credentials={real_user=slave}} blocked reactor thread for 44655.8us
W20260709 13:13:45.395527 29122 leader_election.cc:336] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:45.400395 29048 consensus_peers.cc:597] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:13:45.414494  3097 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:45.414671  3114 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:45.415396  3097 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:45.415411  3114 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:45.417950  3097 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:45.418627  3114 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:45.418708 29048 consensus_peers.cc:597] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:13:45.420629 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:45.423024 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8e6d06054712b0c7cd2c5690ec63" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
I20260709 13:13:45.425535  3112 raft_consensus.cc:493] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ca45420e033847489ad989c1c6eed3ef)
I20260709 13:13:45.426381  3112 raft_consensus.cc:515] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:45.429788  3112 leader_election.cc:290] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:45.443784 29716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:45.440672 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddba9f7f2a6843c480dac4ac4efa4931" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:45.443271 29083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29895dd2e2b6460b8106269935db24f7" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "befda4a6c8344c26b0f139e4e1686b3b" is_pre_election: true
W20260709 13:13:45.453729 28939 debug-util.cc:398] Leaking SignalData structure 0x7b08003bfa80 after lost signal to thread 27904
W20260709 13:13:45.454867 28939 debug-util.cc:398] Leaking SignalData structure 0x7b0800382b00 after lost signal to thread 28942
W20260709 13:13:45.456203 28939 debug-util.cc:398] Leaking SignalData structure 0x7b08003eb4c0 after lost signal to thread 29109
I20260709 13:13:45.456687 29125 leader_election.cc:304] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
W20260709 13:13:45.457258 28939 debug-util.cc:398] Leaking SignalData structure 0x7b08002add00 after lost signal to thread 29183
I20260709 13:13:45.457907  3090 raft_consensus.cc:2749] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:45.458412 28939 debug-util.cc:398] Leaking SignalData structure 0x7b0800156020 after lost signal to thread 29741
I20260709 13:13:45.457901 29125 leader_election.cc:304] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:45.462649  3090 raft_consensus.cc:2749] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:45.472810 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:13:45.499624 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:13:45.500664 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.501338 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.501736 29680 leader_election.cc:336] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:45.502111 29680 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:45.502897  3114 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:45.516302 29682 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:45.517506  3114 raft_consensus.cc:2749] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:45.534335 29122 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.669301 29122 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.844803 29122 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 46168.4us
W20260709 13:13:45.876258 29048 consensus_peers.cc:597] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.876917 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.888082 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.889401 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.893323 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.893827 29048 consensus_peers.cc:597] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.895555 29122 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.905169 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.905910 29122 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:45.908054 29048 consensus_peers.cc:597] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:13:46.124782 27901 tablet_server.cc:196] TabletServer@127.27.63.65:0 shutdown complete.
W20260709 13:13:46.196992 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:46.259709 29048 consensus_peers.cc:597] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:46.268159 29122 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:46.316231 29122 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:46.350023 29122 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:13:46.420701 27901 tablet_server.cc:179] TabletServer@127.27.63.66:0 shutting down...
W20260709 13:13:46.437618 29125 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 403 similar messages]
W20260709 13:13:46.442045 29125 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.457039 29683 consensus_peers.cc:597] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.489077 29683 consensus_peers.cc:597] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.491101 29125 consensus_peers.cc:597] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.506397 29125 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.507232 29122 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:13:46.512497 29125 consensus_peers.cc:597] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.518788 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:13:46.519883 27901 tablet_replica.cc:333] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.520962 27901 raft_consensus.cc:2243] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.521632 27901 raft_consensus.cc:2272] T 8d989a55ed204649a0440142ce55e5ef P befda4a6c8344c26b0f139e4e1686b3b [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.527184 27901 tablet_replica.cc:333] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.527807 27901 raft_consensus.cc:2243] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.528599 27901 raft_consensus.cc:2272] T 9b9b21ce33904ed489b959d5aef0349f P befda4a6c8344c26b0f139e4e1686b3b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.530771 27901 tablet_replica.cc:333] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.531335 27901 raft_consensus.cc:2243] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.531797 27901 raft_consensus.cc:2272] T c141925650a74de58026be545d874618 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.533921 27901 tablet_replica.cc:333] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.534564 27901 raft_consensus.cc:2243] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.535096 27901 raft_consensus.cc:2272] T 00715172c7364d849f262c553c53e5fe P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.537777 27901 tablet_replica.cc:333] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.538565 27901 raft_consensus.cc:2243] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.540416 27901 raft_consensus.cc:2272] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.543130 27901 tablet_replica.cc:333] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.544600 27901 raft_consensus.cc:2243] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:13:46.545341 29125 consensus_peers.cc:597] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.545660 27901 raft_consensus.cc:2272] T 31c5d53651b5426780911903843f0bc3 P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.550803 27901 tablet_replica.cc:333] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.551462 27901 raft_consensus.cc:2243] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.552356 27901 raft_consensus.cc:2272] T 2bf52cc065f943c9a83088bbeea5c794 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.554287 27901 tablet_replica.cc:333] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.554857 27901 raft_consensus.cc:2243] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.555589 27901 raft_consensus.cc:2272] T dafcee70c76341f2a36a461c97f2b3ba P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.557875 27901 tablet_replica.cc:333] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.558460 27901 raft_consensus.cc:2243] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.560083 27901 raft_consensus.cc:2272] T bf1e3918423546bcafc9359aca2ca811 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.562721 27901 tablet_replica.cc:333] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.563527 27901 raft_consensus.cc:2243] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.564787 27901 raft_consensus.cc:2272] T 2bbf1db4b075436695fd521f5a5b2e9d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:46.565447 29125 consensus_peers.cc:597] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.567291 27901 tablet_replica.cc:333] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.567965 27901 raft_consensus.cc:2243] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.568739 27901 raft_consensus.cc:2272] T 94045d5a8709418e82919c9fbf804582 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.571579 27901 tablet_replica.cc:333] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.572443 27901 raft_consensus.cc:2243] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.573678 27901 raft_consensus.cc:2272] T 26c8dceb42c541c8906c424b9bc9bb17 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.576427 27901 tablet_replica.cc:333] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.577226 27901 raft_consensus.cc:2243] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.578089 27901 raft_consensus.cc:2272] T 1db879e214974ab38bc94a40afcf6e1e P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:46.579586 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:46.580662 27901 tablet_replica.cc:333] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.581529 27901 raft_consensus.cc:2243] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.582341 27901 raft_consensus.cc:2272] T 640d00aba1b541e4abb90c63e6b8720d P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:46.584599 29125 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.584993 27901 tablet_replica.cc:333] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.585741 27901 raft_consensus.cc:2243] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.586585 27901 raft_consensus.cc:2272] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.588696 27901 tablet_replica.cc:333] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.589239 27901 raft_consensus.cc:2243] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.590098 27901 raft_consensus.cc:2272] T 8da4f9ce63d54a63a8b9de11ba485699 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.592013 27901 tablet_replica.cc:333] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.592599 27901 raft_consensus.cc:2243] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.594247 27901 raft_consensus.cc:2272] T 824fc688db264856b025a29c651eec78 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.596894 27901 tablet_replica.cc:333] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.597672 27901 raft_consensus.cc:2243] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 8 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.598690 27901 raft_consensus.cc:2272] T 027a3f0877db4e259d94a8e296822644 P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.601465 27901 tablet_replica.cc:333] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.602279 27901 raft_consensus.cc:2243] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.603193 27901 raft_consensus.cc:2272] T d83f8e6d06054712b0c7cd2c5690ec63 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.605768 27901 tablet_replica.cc:333] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.606613 27901 raft_consensus.cc:2243] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.608068 27901 raft_consensus.cc:2272] T 3eaf29b1a5c049b1bc9159a53ce52afb P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:46.608868 29683 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.610853 27901 tablet_replica.cc:333] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.611835 27901 raft_consensus.cc:2243] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.612840 27901 raft_consensus.cc:2272] T e21ca94fafcc4923a671c355b710f10d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.615506 27901 tablet_replica.cc:333] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.616372 27901 raft_consensus.cc:2243] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.617061 27901 raft_consensus.cc:2272] T fa4501421f5146e4a7b79fd8697f8f57 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.619706 27901 tablet_replica.cc:333] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.620539 27901 raft_consensus.cc:2243] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.621162 27901 raft_consensus.cc:2272] T 3d4c87bb2e4a4f2ca684feff1fae17cf P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.623880 27901 tablet_replica.cc:333] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.624751 27901 raft_consensus.cc:2243] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.625710 27901 raft_consensus.cc:2272] T 2185927c34434da38aa9e7776e483b1b P befda4a6c8344c26b0f139e4e1686b3b [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.628494 27901 tablet_replica.cc:333] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.629272 27901 raft_consensus.cc:2243] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.629908 27901 raft_consensus.cc:2272] T e225d0308db94fb9ab850a0b06a63551 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.632715 27901 tablet_replica.cc:333] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.633577 27901 raft_consensus.cc:2243] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.634668 27901 raft_consensus.cc:2272] T 7216d7dc49494630845e9eeb30dc6902 P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.637521 27901 tablet_replica.cc:333] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.638314 27901 raft_consensus.cc:2243] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.639154 27901 raft_consensus.cc:2272] T 5cac0f90a42148b5923aa84c7b7cf00b P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.641660 27901 tablet_replica.cc:333] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.642454 27901 raft_consensus.cc:2243] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.643096 27901 raft_consensus.cc:2272] T 44c8fda62bd34c2f87eb2114b065741e P befda4a6c8344c26b0f139e4e1686b3b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.645717 27901 tablet_replica.cc:333] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.646526 27901 raft_consensus.cc:2243] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:13:46.647291 29125 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.647341 27901 raft_consensus.cc:2272] T 72d50ab8517242a1bb42501ec563c433 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.655665 27901 tablet_replica.cc:333] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
W20260709 13:13:46.656131 29125 consensus_peers.cc:597] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.656610 27901 raft_consensus.cc:2243] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.658077 27901 raft_consensus.cc:2272] T f68cf8a7e5894c679a8bba8d7d2374a5 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.661047 27901 tablet_replica.cc:333] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.661850 27901 raft_consensus.cc:2243] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.662822 27901 raft_consensus.cc:2272] T 0df5a8a9dbd84e3bad34bf60478e7c98 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.665549 27901 tablet_replica.cc:333] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.666615 27901 raft_consensus.cc:2243] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.667822 27901 raft_consensus.cc:2272] T b39693173a844463b5f0947904461289 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.671296 27901 tablet_replica.cc:333] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.672191 27901 raft_consensus.cc:2243] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.673141 27901 raft_consensus.cc:2272] T 57dd7e7308fc43e0b0983412129aa471 P befda4a6c8344c26b0f139e4e1686b3b [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.676024 27901 tablet_replica.cc:333] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.676828 27901 raft_consensus.cc:2243] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.677412 27901 raft_consensus.cc:2272] T 29895dd2e2b6460b8106269935db24f7 P befda4a6c8344c26b0f139e4e1686b3b [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.680326 27901 tablet_replica.cc:333] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.681133 27901 raft_consensus.cc:2243] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.681770 27901 raft_consensus.cc:2272] T ce7f7e5792294b1f99a1bf4ee332a76c P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.684618 27901 tablet_replica.cc:333] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.685387 27901 raft_consensus.cc:2243] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.686311 27901 raft_consensus.cc:2272] T 7fb3976ea75a4cb3b4f543d842c2409c P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.689147 27901 tablet_replica.cc:333] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.689965 27901 raft_consensus.cc:2243] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.690919 27901 raft_consensus.cc:2272] T e1a1cae5d4eb46b288c03a3675d24209 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:46.691478 29683 consensus_peers.cc:597] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.693616 27901 tablet_replica.cc:333] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.694455 27901 raft_consensus.cc:2243] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.695312 27901 raft_consensus.cc:2272] T d8340da85294490fb88f62f80393531f P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.698065 27901 tablet_replica.cc:333] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.698791 27901 raft_consensus.cc:2243] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.699529 27901 raft_consensus.cc:2272] T ea8e44f6f34a4291baf426298bc260e6 P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:46.700171 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.702118 27901 tablet_replica.cc:333] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.702975 27901 raft_consensus.cc:2243] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.703752 27901 raft_consensus.cc:2272] T 3dcde0796c504aa0ad4a5e79df6a169d P befda4a6c8344c26b0f139e4e1686b3b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.706645 27901 tablet_replica.cc:333] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.707427 27901 raft_consensus.cc:2243] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.708309 27901 raft_consensus.cc:2272] T e583be49bed64dd2996afcdcffa95120 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.711207 27901 tablet_replica.cc:333] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.711983 27901 raft_consensus.cc:2243] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.712649 27901 raft_consensus.cc:2272] T 828bc1c562f941c89bdcc43cf79e5ece P befda4a6c8344c26b0f139e4e1686b3b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.715514 27901 tablet_replica.cc:333] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
W20260709 13:13:46.716025 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.716313 27901 raft_consensus.cc:2243] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.717535 27901 raft_consensus.cc:2272] T d012d02d5bf34cbf8f4451c054b36f63 P befda4a6c8344c26b0f139e4e1686b3b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.720377 27901 tablet_replica.cc:333] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.721151 27901 raft_consensus.cc:2243] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.721768 27901 raft_consensus.cc:2272] T 68c287f2818e4a23b258f6c19741c26a P befda4a6c8344c26b0f139e4e1686b3b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.724973 27901 tablet_replica.cc:333] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.725749 27901 raft_consensus.cc:2243] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.726871 27901 raft_consensus.cc:2272] T 6415e058581b4b7eb883c6e96b9fbfcf P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.729589 27901 tablet_replica.cc:333] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.730372 27901 raft_consensus.cc:2243] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.731051 27901 raft_consensus.cc:2272] T 2ed915f0be134d71b36d8fc8f7b5830b P befda4a6c8344c26b0f139e4e1686b3b [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.734612 27901 tablet_replica.cc:333] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.735368 27901 raft_consensus.cc:2243] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:13:46.735061 29683 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:46.736130 27901 raft_consensus.cc:2272] T 99a1616611dd41e49ec3ffc7eb3136a9 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:46.737679 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.739305 27901 tablet_replica.cc:333] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.740113 27901 raft_consensus.cc:2243] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:46.741307 27901 raft_consensus.cc:2272] T 00c2e050cffc43618bffbaaa661fcfa6 P befda4a6c8344c26b0f139e4e1686b3b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:46.744161 27901 tablet_replica.cc:333] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b: stopping tablet replica
I20260709 13:13:46.744840 27901 raft_consensus.cc:2243] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:46.745534 27901 raft_consensus.cc:2272] T 21d163c72e704bd0bd37c74d96761663 P befda4a6c8344c26b0f139e4e1686b3b [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:46.746798 29125 consensus_peers.cc:597] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.747552 29125 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.770944 29125 consensus_peers.cc:597] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.772854 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.787391 29683 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.800848 29180 debug-util.cc:398] Leaking SignalData structure 0x7b080030f640 after lost signal to thread 27904
W20260709 13:13:46.802014 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08003a7a60 after lost signal to thread 28942
W20260709 13:13:46.803295 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3000 after lost signal to thread 29109
W20260709 13:13:46.803622 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:46.804395 29180 debug-util.cc:398] Leaking SignalData structure 0x7b08002f9a60 after lost signal to thread 29183
W20260709 13:13:46.805540 29180 debug-util.cc:398] Leaking SignalData structure 0x7b080017b3e0 after lost signal to thread 29741
W20260709 13:13:46.805778 29683 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.809799 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.846910 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.849210 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.852785 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.856542 29125 consensus_peers.cc:597] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.867089 29125 consensus_peers.cc:597] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.860523 29122 consensus_peers.cc:597] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:13:46.881927 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.893105 29122 consensus_peers.cc:597] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:13:46.942297 29125 consensus_peers.cc:597] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.957379 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.960031 29683 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.960851 29683 consensus_peers.cc:597] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:46.982036 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.039259 29125 consensus_peers.cc:597] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:47.044430 29049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.072492 29683 consensus_peers.cc:597] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:47.084985 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:47.114960 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.140764 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.198118 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.248381 29048 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.251753 29051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.254253 29122 consensus_peers.cc:597] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:47.280898 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.322750 29050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.356644 29122 consensus_peers.cc:597] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:47.389320 29680 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:47.414702 29122 consensus_peers.cc:597] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:47.514525 29122 consensus_peers.cc:597] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:47.565658  3116 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.566246  3116 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.570917  3116 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:47.571244 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
W20260709 13:13:47.576890 29683 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.577334 29683 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:47.578310  3116 raft_consensus.cc:2749] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:47.581919 29680 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:47.583931 29122 consensus_peers.cc:597] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:47.611327  3125 raft_consensus.cc:493] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.611948  3125 raft_consensus.cc:515] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.614264  3125 leader_election.cc:290] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:13:47.622915 29122 leader_election.cc:336] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:47.625245 29125 leader_election.cc:336] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.625787 29125 leader_election.cc:304] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:47.627367  3125 raft_consensus.cc:2749] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:47.652073  3125 raft_consensus.cc:493] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.652619  3125 raft_consensus.cc:515] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.654767  3125 leader_election.cc:290] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
W20260709 13:13:47.660180 29125 leader_election.cc:336] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.666883  3116 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.667479  3116 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.670495  3116 leader_election.cc:290] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:47.672070 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:47.673054  3116 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.677243  3116 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.680536 29155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:47.688980  3125 raft_consensus.cc:493] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.695885  3125 raft_consensus.cc:515] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } attrs { replace: true } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:47.694865 29683 leader_election.cc:336] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.699604 29683 leader_election.cc:304] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:47.700564  3127 raft_consensus.cc:2749] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:47.705165  3116 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:47.707470 29683 leader_election.cc:336] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.709568 29683 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:47.712509  3116 raft_consensus.cc:2749] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:47.715665 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf1e3918423546bcafc9359aca2ca811" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:47.716378 29717 raft_consensus.cc:2468] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
I20260709 13:13:47.711236  3125 leader_election.cc:290] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:47.736546  3119 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.737054  3119 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.739043  3119 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:47.744266 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:47.744880 29717 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 15.
I20260709 13:13:47.760142  3125 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.760780  3125 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.762869  3125 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:47.766865  3127 raft_consensus.cc:493] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.818041  3127 raft_consensus.cc:515] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.820240  3127 leader_election.cc:290] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.771888  3115 raft_consensus.cc:493] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.820835  3115 raft_consensus.cc:515] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.822799  3115 leader_election.cc:290] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.779207  3128 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.786926  3132 raft_consensus.cc:493] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.823717  3128 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.823863  3132 raft_consensus.cc:515] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.826062  3128 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.826252  3132 leader_election.cc:290] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.827459  3128 raft_consensus.cc:493] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.828336  3127 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.828838  3127 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.830924  3127 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
W20260709 13:13:47.813310 29680 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:47.808524  3116 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.771004  3119 raft_consensus.cc:493] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.838553  3116 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.838670  3119 raft_consensus.cc:515] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.840711  3116 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:47.842089  3119 leader_election.cc:290] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.827971  3128 raft_consensus.cc:515] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.844108  3128 leader_election.cc:290] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.814352 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:47.845018  3119 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:47.845446  3119 raft_consensus.cc:493] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.845853  3119 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:47.852401  3132 raft_consensus.cc:493] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.852913  3132 raft_consensus.cc:515] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.854907  3132 leader_election.cc:290] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.859051  3119 raft_consensus.cc:515] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:47.859953 29125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:44167, user_credentials={real_user=slave}} blocked reactor thread for 87311.2us
W20260709 13:13:47.895071 29125 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.873245  3116 raft_consensus.cc:493] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.895799  3116 raft_consensus.cc:515] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.898453  3116 leader_election.cc:290] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.876356  3127 raft_consensus.cc:493] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.877483  3132 raft_consensus.cc:493] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.880002 29150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:47.899459 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:47.899847 29150 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 12.
I20260709 13:13:47.899730  3132 raft_consensus.cc:515] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.900142 29717 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 12.
I20260709 13:13:47.882745 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:47.901813  3132 leader_election.cc:290] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.884254  3125 raft_consensus.cc:493] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.902478  3125 raft_consensus.cc:515] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.904687  3125 leader_election.cc:290] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.904735  3132 raft_consensus.cc:493] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.905484  3132 raft_consensus.cc:515] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.905566  3127 raft_consensus.cc:515] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.907830  3127 leader_election.cc:290] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.907922  3132 leader_election.cc:290] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.912285  3125 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.912796  3125 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.914168  3115 raft_consensus.cc:493] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.914776  3125 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:47.914644  3115 raft_consensus.cc:515] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:47.916733  3115 leader_election.cc:290] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.919891 29682 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:47.920706  3127 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:47.921120  3127 raft_consensus.cc:493] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.921519  3127 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:47.937162  3134 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.937819  3134 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.939941  3134 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:47.872561  3128 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.946242  3128 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:47.894080 29680 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:47.947641 29680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:43941, user_credentials={real_user=slave}} blocked reactor thread for 53589.5us
W20260709 13:13:47.948165 29680 leader_election.cc:336] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:47.948737 29680 leader_election.cc:336] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:47.954506  3128 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:47.956234 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256" is_pre_election: true
I20260709 13:13:47.956853 29158 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 14.
I20260709 13:13:47.958242 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:47.958926  3128 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:47.959299  3128 raft_consensus.cc:493] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.959666  3128 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.961225  3134 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.961884  3134 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:47.964294 29683 leader_election.cc:336] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.964742 29683 leader_election.cc:304] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:47.966917  3134 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:47.968201 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:47.945312  3127 raft_consensus.cc:515] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.972553  3127 leader_election.cc:290] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:47.943568  3119 leader_election.cc:290] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:47.973958  3116 raft_consensus.cc:2749] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:47.974094 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b4bfdf0b2488586aa6cbe1ca64200" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:47.978524 29125 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
W20260709 13:13:47.980810 29683 leader_election.cc:336] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.985474  3137 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:47.987998 29158 raft_consensus.cc:3060] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:47.988814  3137 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:47.991187  3137 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:47.991737 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:47.994194 29683 leader_election.cc:336] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.006603 29683 leader_election.cc:336] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:47.995275 29158 raft_consensus.cc:2468] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 13.
I20260709 13:13:48.002274 29715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:47.994652  3127 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:48.008643 29683 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 243d4ae4a4ae48b2979e32874472d256, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.022285 29715 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:13:48.022840  3137 raft_consensus.cc:2749] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:48.022735 29683 leader_election.cc:336] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.023874 29683 leader_election.cc:304] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.024547  3137 raft_consensus.cc:2749] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:48.008788  3119 raft_consensus.cc:2764] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:13:48.009473 29682 leader_election.cc:304] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:48.021468  3128 raft_consensus.cc:515] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:48.029134 29717 raft_consensus.cc:2393] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 243d4ae4a4ae48b2979e32874472d256 in current term 15: Already voted for candidate ca45420e033847489ad989c1c6eed3ef in this term.
I20260709 13:13:48.031800  3137 raft_consensus.cc:2804] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:48.032625  3137 raft_consensus.cc:697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:13:48.033735  3137 consensus_queue.cc:237] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:48.020922 29680 leader_election.cc:336] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.039089 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b39693173a844463b5f0947904461289" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:48.039692 29717 raft_consensus.cc:3060] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:48.046576 29717 raft_consensus.cc:2468] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 16.
I20260709 13:13:48.049203  3128 leader_election.cc:290] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:48.050793 29158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6415e058581b4b7eb883c6e96b9fbfcf" candidate_uuid: "ca45420e033847489ad989c1c6eed3ef" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "243d4ae4a4ae48b2979e32874472d256"
I20260709 13:13:48.051379 29158 raft_consensus.cc:3060] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:48.051807 29125 leader_election.cc:336] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.050786 29680 leader_election.cc:336] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:48.052845 29680 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:48.057731 29158 raft_consensus.cc:2468] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca45420e033847489ad989c1c6eed3ef in term 15.
I20260709 13:13:48.058866 29682 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:48.060613  3127 raft_consensus.cc:2804] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:48.061924  3127 raft_consensus.cc:697] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 15 LEADER]: Becoming Leader. State: Replica: ca45420e033847489ad989c1c6eed3ef, State: Running, Role: LEADER
I20260709 13:13:48.062779  3127 consensus_queue.cc:237] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:48.068928 29125 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.073928 29125 leader_election.cc:336] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.078249 28901 catalog_manager.cc:5697] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 12 to 13, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 13 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:48.079195 29125 leader_election.cc:336] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.079702 29125 leader_election.cc:336] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.081851 29125 leader_election.cc:336] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.088976 29680 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:13:48.089653 29680 leader_election.cc:336] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:48.094810 29680 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:13:48.095947 29683 leader_election.cc:336] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.096513 29683 leader_election.cc:304] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.099339  3128 raft_consensus.cc:2749] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:48.102512 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef" is_pre_election: true
I20260709 13:13:48.103160 29717 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 8.
W20260709 13:13:48.104202 29683 leader_election.cc:336] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.104624 29683 leader_election.cc:304] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.107863  3119 raft_consensus.cc:493] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:48.111935  3119 raft_consensus.cc:515] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } }
I20260709 13:13:48.114619  3119 leader_election.cc:290] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:48.116285  3127 raft_consensus.cc:2749] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:48.119115 29683 leader_election.cc:336] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.119889 29683 leader_election.cc:336] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.120661 29683 leader_election.cc:336] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.121016 29683 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.121745  3127 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:48.121944 29683 leader_election.cc:336] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.122929 29125 leader_election.cc:336] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.123457 29125 leader_election.cc:336] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.123910 29125 leader_election.cc:336] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.124501 29125 leader_election.cc:336] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.124964 29125 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.125387 29125 leader_election.cc:336] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
W20260709 13:13:48.125837 29125 leader_election.cc:336] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.130487 29125 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.131212  3134 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:48.131580  3134 raft_consensus.cc:493] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:48.131940  3134 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:48.134518 29125 leader_election.cc:304] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: befda4a6c8344c26b0f139e4e1686b3b, ca45420e033847489ad989c1c6eed3ef
I20260709 13:13:48.135661 29125 leader_election.cc:304] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.135823  3119 raft_consensus.cc:2749] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:48.136564  3132 raft_consensus.cc:2804] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:48.136814 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 
I20260709 13:13:48.138098  3125 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:13:48.138338 29125 leader_election.cc:336] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.138468  3125 raft_consensus.cc:493] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Starting leader election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:48.139412  3125 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.139894  3134 raft_consensus.cc:515] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:48.137037  3132 raft_consensus.cc:697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:48.141937  3134 leader_election.cc:290] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:48.142730  3132 consensus_queue.cc:237] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:48.144671 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bbf1db4b075436695fd521f5a5b2e9d" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
W20260709 13:13:48.145007 29125 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.145244 29717 raft_consensus.cc:3060] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:48.152863 28901 catalog_manager.cc:5697] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef reported cstate change: term changed from 14 to 15, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to ca45420e033847489ad989c1c6eed3ef (127.27.63.68). New cstate: current_term: 15 leader_uuid: "ca45420e033847489ad989c1c6eed3ef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:48.154080 29717 raft_consensus.cc:2468] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 17.
W20260709 13:13:48.154937 29125 leader_election.cc:336] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.163584 29125 leader_election.cc:304] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.165189  3134 raft_consensus.cc:2804] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:48.165652  3134 raft_consensus.cc:697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:48.166425  3134 consensus_queue.cc:237] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:48.168267 28906 catalog_manager.cc:5697] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 15 to 16, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 16 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:48.187234 29680 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:48.187716  3125 raft_consensus.cc:515] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:48.194145  3125 leader_election.cc:290] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Requested vote from peers befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939), ca45420e033847489ad989c1c6eed3ef (127.27.63.68:44167)
I20260709 13:13:48.196203 29717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027a3f0877db4e259d94a8e296822644" candidate_uuid: "243d4ae4a4ae48b2979e32874472d256" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ca45420e033847489ad989c1c6eed3ef"
I20260709 13:13:48.196843 29717 raft_consensus.cc:3060] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.202845 28906 catalog_manager.cc:5697] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 16 to 17, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 17 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:48.206494 29717 raft_consensus.cc:2468] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243d4ae4a4ae48b2979e32874472d256 in term 9.
W20260709 13:13:48.207768 29125 leader_election.cc:336] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:48.209584 29125 leader_election.cc:304] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:48.212771  3125 raft_consensus.cc:2804] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:48.213284  3125 raft_consensus.cc:697] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Becoming Leader. State: Replica: 243d4ae4a4ae48b2979e32874472d256, State: Running, Role: LEADER
I20260709 13:13:48.214174  3125 consensus_queue.cc:237] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
W20260709 13:13:48.214958 29680 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:48.227023 28906 catalog_manager.cc:5697] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 reported cstate change: term changed from 8 to 9, leader changed from befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66) to 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67). New cstate: current_term: 9 leader_uuid: "243d4ae4a4ae48b2979e32874472d256" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:48.228369 29680 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:48.280719 27901 tablet_server.cc:196] TabletServer@127.27.63.66:0 shutdown complete.
I20260709 13:13:48.400863 27901 tablet_server.cc:179] TabletServer@127.27.63.67:0 shutting down...
W20260709 13:13:48.431066 29680 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:13:48.470067 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:13:48.471115 27901 tablet_replica.cc:333] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.471855 27901 raft_consensus.cc:2243] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.472731 27901 raft_consensus.cc:2272] T 6a08e186ea0749ae979329103591a933 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.475090 27901 tablet_replica.cc:333] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.475701 27901 raft_consensus.cc:2243] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.476224 27901 raft_consensus.cc:2272] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.478267 27901 tablet_replica.cc:333] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.478848 27901 raft_consensus.cc:2243] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.479375 27901 raft_consensus.cc:2272] T c32aba72953748b4bb7a95efc76501ff P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.481352 27901 tablet_replica.cc:333] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.482051 27901 raft_consensus.cc:2243] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.482569 27901 raft_consensus.cc:2272] T 6415e058581b4b7eb883c6e96b9fbfcf P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.484558 27901 tablet_replica.cc:333] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.485143 27901 raft_consensus.cc:2243] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.485884 27901 raft_consensus.cc:2272] T f97f68627cfa4585b418a3c090e44c14 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.487948 27901 tablet_replica.cc:333] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.488757 27901 raft_consensus.cc:2243] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.489368 27901 raft_consensus.cc:2272] T 2ed915f0be134d71b36d8fc8f7b5830b P 243d4ae4a4ae48b2979e32874472d256 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.491758 27901 tablet_replica.cc:333] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.492565 27901 raft_consensus.cc:2243] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.493394 27901 raft_consensus.cc:2272] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.495527 27901 tablet_replica.cc:333] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.496194 27901 raft_consensus.cc:2243] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.497052 27901 raft_consensus.cc:2272] T 49241be91ca743cbac56eb24218ff775 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.499529 27901 tablet_replica.cc:333] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.500362 27901 raft_consensus.cc:2243] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.500953 27901 raft_consensus.cc:2272] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.503058 27901 tablet_replica.cc:333] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.503659 27901 raft_consensus.cc:2243] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.504199 27901 raft_consensus.cc:2272] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.506064 27901 tablet_replica.cc:333] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.506639 27901 raft_consensus.cc:2243] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.507326 27901 raft_consensus.cc:2272] T 31c5d53651b5426780911903843f0bc3 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.509588 27901 tablet_replica.cc:333] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.510195 27901 raft_consensus.cc:2243] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.510766 27901 raft_consensus.cc:2272] T 2bf52cc065f943c9a83088bbeea5c794 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.513173 27901 tablet_replica.cc:333] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.513715 27901 raft_consensus.cc:2243] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.514810 27901 raft_consensus.cc:2272] T 640d00aba1b541e4abb90c63e6b8720d P 243d4ae4a4ae48b2979e32874472d256 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.517280 27901 tablet_replica.cc:333] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.517966 27901 raft_consensus.cc:2243] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.518846 27901 raft_consensus.cc:2272] T 1db879e214974ab38bc94a40afcf6e1e P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:48.519568 29683 consensus_peers.cc:597] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:48.521230 27901 tablet_replica.cc:333] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.521961 27901 raft_consensus.cc:2243] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.522837 27901 pending_rounds.cc:70] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Trying to abort 1 pending ops.
I20260709 13:13:48.523058 27901 pending_rounds.cc:77] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256: Aborting op as it isn't in flight: id { term: 17 index: 15 } timestamp: 7305637184179695616 op_type: NO_OP noop_request { }
I20260709 13:13:48.523455 27901 raft_consensus.cc:2889] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:13:48.523756 27901 raft_consensus.cc:2272] T 2bbf1db4b075436695fd521f5a5b2e9d P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.525847 27901 tablet_replica.cc:333] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.526705 27901 raft_consensus.cc:2243] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.527246 27901 raft_consensus.cc:2272] T 94045d5a8709418e82919c9fbf804582 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.529248 27901 tablet_replica.cc:333] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.529814 27901 raft_consensus.cc:2243] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.530268 27901 raft_consensus.cc:2272] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.532307 27901 tablet_replica.cc:333] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.532871 27901 raft_consensus.cc:2243] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.533725 27901 raft_consensus.cc:2272] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.535804 27901 tablet_replica.cc:333] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.536415 27901 raft_consensus.cc:2243] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.537170 27901 pending_rounds.cc:70] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Trying to abort 1 pending ops.
I20260709 13:13:48.537385 27901 pending_rounds.cc:77] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256: Aborting op as it isn't in flight: id { term: 9 index: 9 } timestamp: 7305637184379428864 op_type: NO_OP noop_request { }
I20260709 13:13:48.537689 27901 raft_consensus.cc:2889] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 9 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:13:48.537972 27901 raft_consensus.cc:2272] T 027a3f0877db4e259d94a8e296822644 P 243d4ae4a4ae48b2979e32874472d256 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.540349 27901 tablet_replica.cc:333] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.541285 27901 raft_consensus.cc:2243] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.541864 27901 raft_consensus.cc:2272] T 342b8b8f6e7742c897bcd3495fecb00a P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.543987 27901 tablet_replica.cc:333] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.544824 27901 raft_consensus.cc:2243] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.545682 27901 raft_consensus.cc:2272] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.549420 27901 tablet_replica.cc:333] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.550015 27901 raft_consensus.cc:2243] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.550453 27901 raft_consensus.cc:2272] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.552480 27901 tablet_replica.cc:333] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.553377 27901 raft_consensus.cc:2243] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.554225 27901 raft_consensus.cc:2272] T 72d50ab8517242a1bb42501ec563c433 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.556900 27901 tablet_replica.cc:333] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
W20260709 13:13:48.557242 29682 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:48.558010 27901 raft_consensus.cc:2243] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.559031 27901 raft_consensus.cc:2272] T 44c8fda62bd34c2f87eb2114b065741e P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.561841 27901 tablet_replica.cc:333] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.562754 27901 raft_consensus.cc:2243] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.563988 27901 raft_consensus.cc:2272] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:48.564191 29682 consensus_peers.cc:597] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:48.567304 27901 tablet_replica.cc:333] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.568277 27901 raft_consensus.cc:2243] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.568996 27901 raft_consensus.cc:2272] T ce7f7e5792294b1f99a1bf4ee332a76c P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:48.570884 29682 consensus_peers.cc:597] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:48.572343 27901 tablet_replica.cc:333] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.573222 27901 raft_consensus.cc:2243] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.573729 27901 raft_consensus.cc:2272] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.576475 27901 tablet_replica.cc:333] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.577374 27901 raft_consensus.cc:2243] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.578120 27901 raft_consensus.cc:2272] T eb070bde5f3744eb90ee5cf3dec8fcd4 P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:48.579222 29682 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:48.580660 27901 tablet_replica.cc:333] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.581295 27901 raft_consensus.cc:2243] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.581888 27901 raft_consensus.cc:2272] T 29895dd2e2b6460b8106269935db24f7 P 243d4ae4a4ae48b2979e32874472d256 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.584344 27901 tablet_replica.cc:333] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.585158 27901 raft_consensus.cc:2243] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.586052 27901 pending_rounds.cc:70] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Trying to abort 1 pending ops.
W20260709 13:13:48.586047 29682 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:48.586395 27901 pending_rounds.cc:77] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256: Aborting op as it isn't in flight: id { term: 16 index: 14 } timestamp: 7305637184081641472 op_type: NO_OP noop_request { }
I20260709 13:13:48.586804 27901 raft_consensus.cc:2889] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:13:48.587108 27901 raft_consensus.cc:2272] T b39693173a844463b5f0947904461289 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.589391 27901 tablet_replica.cc:333] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.589969 27901 raft_consensus.cc:2243] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.590642 27901 raft_consensus.cc:2272] T d8340da85294490fb88f62f80393531f P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.592671 27901 tablet_replica.cc:333] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.593351 27901 raft_consensus.cc:2243] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.593993 27901 raft_consensus.cc:2272] T ea8e44f6f34a4291baf426298bc260e6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.595888 27901 tablet_replica.cc:333] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.596481 27901 raft_consensus.cc:2243] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.597239 27901 raft_consensus.cc:2272] T 3dcde0796c504aa0ad4a5e79df6a169d P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.599627 27901 tablet_replica.cc:333] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.600292 27901 raft_consensus.cc:2243] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.600855 27901 raft_consensus.cc:2272] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.603005 27901 tablet_replica.cc:333] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.603554 27901 raft_consensus.cc:2243] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.604125 27901 raft_consensus.cc:2272] T 3aa255ba50b343eca7717515ec37d959 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.606101 27901 tablet_replica.cc:333] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.606726 27901 raft_consensus.cc:2243] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.607815 27901 raft_consensus.cc:2272] T d012d02d5bf34cbf8f4451c054b36f63 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:48.608312 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.610106 27901 tablet_replica.cc:333] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.610926 27901 raft_consensus.cc:2243] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.611521 27901 raft_consensus.cc:2272] T 68c287f2818e4a23b258f6c19741c26a P 243d4ae4a4ae48b2979e32874472d256 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.613710 27901 tablet_replica.cc:333] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.614260 27901 raft_consensus.cc:2243] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.614995 27901 raft_consensus.cc:2272] T 99a1616611dd41e49ec3ffc7eb3136a9 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.616861 27901 tablet_replica.cc:333] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.617399 27901 raft_consensus.cc:2243] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.618278 27901 raft_consensus.cc:2272] T fa4501421f5146e4a7b79fd8697f8f57 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.620177 27901 tablet_replica.cc:333] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.620870 27901 raft_consensus.cc:2243] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.621642 27901 raft_consensus.cc:2272] T e225d0308db94fb9ab850a0b06a63551 P 243d4ae4a4ae48b2979e32874472d256 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.624305 27901 tablet_replica.cc:333] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.625037 27901 raft_consensus.cc:2243] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:13:48.625386 29123 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.626118 27901 raft_consensus.cc:2272] T 3d4c87bb2e4a4f2ca684feff1fae17cf P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:48.627301 29683 consensus_peers.cc:597] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:48.628898 27901 tablet_replica.cc:333] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.629787 27901 raft_consensus.cc:2243] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.630445 27901 raft_consensus.cc:2272] T 86e87ad9e9a5439d890ebaf60f043eb6 P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.633144 27901 tablet_replica.cc:333] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.633939 27901 raft_consensus.cc:2243] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.634722 27901 raft_consensus.cc:2272] T 828bc1c562f941c89bdcc43cf79e5ece P 243d4ae4a4ae48b2979e32874472d256 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.636972 27901 tablet_replica.cc:333] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.637696 27901 raft_consensus.cc:2243] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.638531 27901 raft_consensus.cc:2272] T e583be49bed64dd2996afcdcffa95120 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:48.639482 29682 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:48.641222 27901 tablet_replica.cc:333] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.642052 27901 raft_consensus.cc:2243] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.642702 27901 raft_consensus.cc:2272] T ab75774d33164f0b9c0269be22426768 P 243d4ae4a4ae48b2979e32874472d256 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.645062 27901 tablet_replica.cc:333] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.645596 27901 raft_consensus.cc:2243] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.646096 27901 raft_consensus.cc:2272] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.648211 27901 tablet_replica.cc:333] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.648662 27901 raft_consensus.cc:2243] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:48.649269 27901 raft_consensus.cc:2272] T 5cac0f90a42148b5923aa84c7b7cf00b P 243d4ae4a4ae48b2979e32874472d256 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:48.650965 27901 tablet_replica.cc:333] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256: stopping tablet replica
I20260709 13:13:48.651401 27901 raft_consensus.cc:2243] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:48.651844 27901 raft_consensus.cc:2272] T ddba9f7f2a6843c480dac4ac4efa4931 P 243d4ae4a4ae48b2979e32874472d256 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:48.660995 29125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:48.662897 29122 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:48.670269 29682 consensus_peers.cc:597] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.678848 29682 consensus_peers.cc:597] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.684155 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:48.685770 29122 leader_election.cc:336] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.686278 29122 leader_election.cc:304] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.686928 29122 raft_consensus.cc:2686] T 7216d7dc49494630845e9eeb30dc6902 P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.688666 29122 leader_election.cc:336] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.689065 29122 leader_election.cc:304] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.689709 29122 raft_consensus.cc:2686] T 0df5a8a9dbd84e3bad34bf60478e7c98 P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.690862 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:48.691479 29122 leader_election.cc:336] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.691918 29122 leader_election.cc:304] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.692492 29122 raft_consensus.cc:2686] T 5ec27c4bcd6b4eb3a5165f616c4ad641 P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.693074 29122 leader_election.cc:336] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.693456 29122 leader_election.cc:304] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.693992 29122 raft_consensus.cc:2686] T 9b9b21ce33904ed489b959d5aef0349f P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.694758 29122 leader_election.cc:336] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.695120 29122 leader_election.cc:304] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.695672 29122 raft_consensus.cc:2686] T 26c8dceb42c541c8906c424b9bc9bb17 P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.697291 29122 leader_election.cc:336] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.697682 29122 leader_election.cc:304] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.698284 29122 raft_consensus.cc:2686] T e21ca94fafcc4923a671c355b710f10d P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.699877 29122 leader_election.cc:336] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.700285 29122 leader_election.cc:304] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.700834 29122 raft_consensus.cc:2686] T 7fb3976ea75a4cb3b4f543d842c2409c P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.702589 29122 leader_election.cc:336] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:48.702975 29122 leader_election.cc:304] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.703496 29122 raft_consensus.cc:2686] T 00c2e050cffc43618bffbaaa661fcfa6 P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.705276 29122 leader_election.cc:336] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
W20260709 13:13:48.706840 29124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.705703 29122 leader_election.cc:304] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.720149 29122 raft_consensus.cc:2686] T f68cf8a7e5894c679a8bba8d7d2374a5 P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.727222 29682 consensus_peers.cc:597] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.729903 27901 leader_election.cc:336] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Aborted: reactor is shutting down (error 108)
I20260709 13:13:48.730329 27901 leader_election.cc:304] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256, ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.730774 27901 raft_consensus.cc:2686] T bf1e3918423546bcafc9359aca2ca811 P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.732223 27901 leader_election.cc:336] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Aborted: reactor is shutting down (error 108)
I20260709 13:13:48.732672 27901 leader_election.cc:304] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243d4ae4a4ae48b2979e32874472d256; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
W20260709 13:13:48.733073 27901 raft_consensus.cc:2686] T 3eaf29b1a5c049b1bc9159a53ce52afb P 243d4ae4a4ae48b2979e32874472d256: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:13:48.766502 29682 consensus_peers.cc:597] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.67:40957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.778256 29682 consensus_peers.cc:597] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.67:40957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.787142 29682 consensus_peers.cc:597] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.67:40957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.794766 29682 consensus_peers.cc:597] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.67:40957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.873016 29680 consensus_peers.cc:597] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:13:48.882357 29682 consensus_peers.cc:597] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.67:40957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.888422 29682 consensus_peers.cc:597] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef -> Peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Couldn't send request to peer 243d4ae4a4ae48b2979e32874472d256. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.67:40957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:48.932163 29680 consensus_peers.cc:597] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:48.959201 29683 consensus_peers.cc:597] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:49.125145 29680 consensus_peers.cc:597] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:13:49.232573 29683 consensus_peers.cc:597] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:49.252233 29683 consensus_peers.cc:597] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:49.350359 29683 consensus_peers.cc:597] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:49.423316 27901 tablet_server.cc:196] TabletServer@127.27.63.67:0 shutdown complete.
W20260709 13:13:49.425457 29683 consensus_peers.cc:597] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:49.470695 29683 consensus_peers.cc:597] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:49.471381 29680 consensus_peers.cc:597] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef -> Peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Couldn't send request to peer 06ae4197fd784525b695a9b16c8094ea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:13:49.517107 29683 consensus_peers.cc:597] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:49.554086 29683 consensus_peers.cc:597] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:49.558877 27901 tablet_server.cc:179] TabletServer@127.27.63.68:0 shutting down...
W20260709 13:13:49.600914 29683 consensus_peers.cc:597] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:49.615860 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:13:49.617504 27901 tablet_replica.cc:333] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.618422 27901 raft_consensus.cc:2243] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.619092 27901 raft_consensus.cc:2272] T bf1e3918423546bcafc9359aca2ca811 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.621978 27901 tablet_replica.cc:333] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.622757 27901 raft_consensus.cc:2243] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.623317 27901 raft_consensus.cc:2272] T 640d00aba1b541e4abb90c63e6b8720d P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.625892 27901 tablet_replica.cc:333] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.626638 27901 raft_consensus.cc:2243] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.627419 27901 raft_consensus.cc:2272] T e1a1cae5d4eb46b288c03a3675d24209 P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.630179 27901 tablet_replica.cc:333] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.630992 27901 raft_consensus.cc:2243] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.631974 27901 raft_consensus.cc:2272] T 2185927c34434da38aa9e7776e483b1b P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.632174  3128 raft_consensus.cc:493] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:49.632810  3128 raft_consensus.cc:515] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:49.634473 27901 tablet_replica.cc:333] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.634989  3128 leader_election.cc:290] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:49.635313 27901 raft_consensus.cc:2243] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.636541 27901 raft_consensus.cc:2272] T ab75774d33164f0b9c0269be22426768 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.639101 27901 tablet_replica.cc:333] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
W20260709 13:13:49.639365 29680 leader_election.cc:336] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:49.639904 27901 raft_consensus.cc:2243] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.639875  3128 raft_consensus.cc:493] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election (detected failure of leader befda4a6c8344c26b0f139e4e1686b3b)
I20260709 13:13:49.640703  3128 raft_consensus.cc:515] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "befda4a6c8344c26b0f139e4e1686b3b" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 35939 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:49.641126 27901 raft_consensus.cc:2272] T 2ed915f0be134d71b36d8fc8f7b5830b P ca45420e033847489ad989c1c6eed3ef [term 10 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:49.641175 29683 leader_election.cc:336] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:49.641786 29683 leader_election.cc:304] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:49.642490  3127 raft_consensus.cc:2749] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:49.643975 27901 tablet_replica.cc:333] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.644863 27901 raft_consensus.cc:2243] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.645498  3128 leader_election.cc:290] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939)
I20260709 13:13:49.645651 27901 raft_consensus.cc:2272] T 824fc688db264856b025a29c651eec78 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:49.647964 29680 leader_election.cc:336] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:49.648386 27901 tablet_replica.cc:333] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.649173 27901 raft_consensus.cc:2243] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.650362 27901 raft_consensus.cc:2272] T 6a08e186ea0749ae979329103591a933 P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:49.652184 29683 leader_election.cc:336] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111)
I20260709 13:13:49.652580 29683 leader_election.cc:304] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, befda4a6c8344c26b0f139e4e1686b3b
I20260709 13:13:49.653156 27901 tablet_replica.cc:333] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.653168  3128 raft_consensus.cc:2749] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:49.653405 29683 consensus_peers.cc:597] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef -> Peer befda4a6c8344c26b0f139e4e1686b3b (127.27.63.66:35939): Couldn't send request to peer befda4a6c8344c26b0f139e4e1686b3b. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:35939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:49.654086 27901 raft_consensus.cc:2243] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.654665 27901 raft_consensus.cc:2272] T 8da4f9ce63d54a63a8b9de11ba485699 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.657160 27901 tablet_replica.cc:333] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.657886 27901 raft_consensus.cc:2243] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.658798 27901 raft_consensus.cc:2272] T 342b8b8f6e7742c897bcd3495fecb00a P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.661278 27901 tablet_replica.cc:333] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.661972 27901 raft_consensus.cc:2243] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 8 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.662876 27901 raft_consensus.cc:2272] T 00715172c7364d849f262c553c53e5fe P ca45420e033847489ad989c1c6eed3ef [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.665253 27901 tablet_replica.cc:333] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.665952 27901 raft_consensus.cc:2243] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.666940 27901 raft_consensus.cc:2272] T 8d989a55ed204649a0440142ce55e5ef P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.669317 27901 tablet_replica.cc:333] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.669991 27901 raft_consensus.cc:2243] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.670854 27901 pending_rounds.cc:70] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Trying to abort 1 pending ops.
I20260709 13:13:49.671069 27901 pending_rounds.cc:77] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef: Aborting op as it isn't in flight: id { term: 15 index: 11 } timestamp: 7305637183758495744 op_type: NO_OP noop_request { }
I20260709 13:13:49.671455 27901 raft_consensus.cc:2889] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 15 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:13:49.671773 27901 raft_consensus.cc:2272] T 6415e058581b4b7eb883c6e96b9fbfcf P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.674088 27901 tablet_replica.cc:333] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.674783 27901 raft_consensus.cc:2243] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.675309 27901 raft_consensus.cc:2272] T 57dd7e7308fc43e0b0983412129aa471 P ca45420e033847489ad989c1c6eed3ef [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.677776 27901 tablet_replica.cc:333] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.678496 27901 raft_consensus.cc:2243] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.678998 27901 raft_consensus.cc:2272] T 027a3f0877db4e259d94a8e296822644 P ca45420e033847489ad989c1c6eed3ef [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.681643 27901 tablet_replica.cc:333] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.682314 27901 raft_consensus.cc:2243] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.683301 27901 raft_consensus.cc:2272] T ddba9f7f2a6843c480dac4ac4efa4931 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.685599  3128 raft_consensus.cc:493] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:49.685756 27901 tablet_replica.cc:333] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.686115  3128 raft_consensus.cc:515] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:49.686633 27901 raft_consensus.cc:2243] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.687328 27901 raft_consensus.cc:2272] T d012d02d5bf34cbf8f4451c054b36f63 P ca45420e033847489ad989c1c6eed3ef [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.690056 27901 tablet_replica.cc:333] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.690354  3128 leader_election.cc:290] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:49.690827 27901 raft_consensus.cc:2243] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.691452 27901 raft_consensus.cc:2272] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:49.692629 29682 leader_election.cc:336] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Network error: Client connection negotiation failed: client connection to 127.27.63.67:40957: connect: Connection refused (error 111)
W20260709 13:13:49.693656 29680 leader_election.cc:336] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:49.694044 27901 tablet_replica.cc:333] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.694104 29680 leader_election.cc:304] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:49.694934 27901 raft_consensus.cc:2243] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Raft consensus shutting down.
W20260709 13:13:49.694943 29680 raft_consensus.cc:2686] T f97f68627cfa4585b418a3c090e44c14 P ca45420e033847489ad989c1c6eed3ef: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:13:49.695914 27901 pending_rounds.cc:70] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Trying to abort 1 pending ops.
I20260709 13:13:49.696163 27901 pending_rounds.cc:77] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef: Aborting op as it isn't in flight: id { term: 13 index: 12 } timestamp: 7305637183636164608 op_type: NO_OP noop_request { }
I20260709 13:13:49.696520 27901 raft_consensus.cc:2889] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:13:49.696902 27901 raft_consensus.cc:2272] T 8f1b4bfdf0b2488586aa6cbe1ca64200 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.699427 27901 tablet_replica.cc:333] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.700222 27901 raft_consensus.cc:2243] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.701215 27901 raft_consensus.cc:2272] T 68c287f2818e4a23b258f6c19741c26a P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.703600 27901 tablet_replica.cc:333] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.704329 27901 raft_consensus.cc:2243] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.704874 27901 raft_consensus.cc:2272] T b39693173a844463b5f0947904461289 P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.707305 27901 tablet_replica.cc:333] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.708015 27901 raft_consensus.cc:2243] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.708961 27901 raft_consensus.cc:2272] T 29895dd2e2b6460b8106269935db24f7 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.711364 27901 tablet_replica.cc:333] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.712028 27901 raft_consensus.cc:2243] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.712949 27901 raft_consensus.cc:2272] T 94045d5a8709418e82919c9fbf804582 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.715190 27901 tablet_replica.cc:333] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.715849 27901 raft_consensus.cc:2243] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.716820 27901 raft_consensus.cc:2272] T c141925650a74de58026be545d874618 P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.719244 27901 tablet_replica.cc:333] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.719929 27901 raft_consensus.cc:2243] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.720878 27901 raft_consensus.cc:2272] T 21d163c72e704bd0bd37c74d96761663 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.723161 27901 tablet_replica.cc:333] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.723892 27901 raft_consensus.cc:2243] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.724493 27901 raft_consensus.cc:2272] T fa4501421f5146e4a7b79fd8697f8f57 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.727070 27901 tablet_replica.cc:333] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.727768 27901 raft_consensus.cc:2243] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.728346 27901 raft_consensus.cc:2272] T 3d4c87bb2e4a4f2ca684feff1fae17cf P ca45420e033847489ad989c1c6eed3ef [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.730707 27901 tablet_replica.cc:333] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.731377 27901 raft_consensus.cc:2243] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.732306 27901 raft_consensus.cc:2272] T 86e87ad9e9a5439d890ebaf60f043eb6 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.734282  3128 raft_consensus.cc:493] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 243d4ae4a4ae48b2979e32874472d256)
I20260709 13:13:49.734640 27901 tablet_replica.cc:333] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.734731  3128 raft_consensus.cc:515] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "06ae4197fd784525b695a9b16c8094ea" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 43941 } } peers { permanent_uuid: "243d4ae4a4ae48b2979e32874472d256" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 40957 } } peers { permanent_uuid: "ca45420e033847489ad989c1c6eed3ef" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 44167 } attrs { promote: false } }
I20260709 13:13:49.735451 27901 raft_consensus.cc:2243] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.736363 27901 raft_consensus.cc:2272] T ce7f7e5792294b1f99a1bf4ee332a76c P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.738701 27901 tablet_replica.cc:333] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.739382 27901 raft_consensus.cc:2243] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.739431  3128 leader_election.cc:290] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941), 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957)
I20260709 13:13:49.740491 27901 raft_consensus.cc:2272] T eb070bde5f3744eb90ee5cf3dec8fcd4 P ca45420e033847489ad989c1c6eed3ef [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:49.741770 29682 leader_election.cc:336] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 243d4ae4a4ae48b2979e32874472d256 (127.27.63.67:40957): Network error: Client connection negotiation failed: client connection to 127.27.63.67:40957: connect: Connection refused (error 111)
I20260709 13:13:49.743000 27901 tablet_replica.cc:333] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
W20260709 13:13:49.742862 29680 leader_election.cc:336] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:49.743346 29680 leader_election.cc:304] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca45420e033847489ad989c1c6eed3ef; no voters: 06ae4197fd784525b695a9b16c8094ea, 243d4ae4a4ae48b2979e32874472d256
I20260709 13:13:49.743554 27901 raft_consensus.cc:2243] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.744220 27901 raft_consensus.cc:2272] T dafcee70c76341f2a36a461c97f2b3ba P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.744213  3128 raft_consensus.cc:2749] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:49.746287 27901 tablet_replica.cc:333] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.746830 27901 raft_consensus.cc:2243] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.747558 27901 raft_consensus.cc:2272] T 3aa255ba50b343eca7717515ec37d959 P ca45420e033847489ad989c1c6eed3ef [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.749400 27901 tablet_replica.cc:333] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.749861 27901 raft_consensus.cc:2243] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.750291 27901 raft_consensus.cc:2272] T d83f8e6d06054712b0c7cd2c5690ec63 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.751952 27901 tablet_replica.cc:333] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.752451 27901 raft_consensus.cc:2243] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:49.753149 27901 raft_consensus.cc:2272] T c32aba72953748b4bb7a95efc76501ff P ca45420e033847489ad989c1c6eed3ef [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.754920 27901 tablet_replica.cc:333] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.755395 27901 raft_consensus.cc:2243] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.755908 27901 raft_consensus.cc:2272] T 2bbf1db4b075436695fd521f5a5b2e9d P ca45420e033847489ad989c1c6eed3ef [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:49.757768 27901 tablet_replica.cc:333] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef: stopping tablet replica
I20260709 13:13:49.758170 27901 raft_consensus.cc:2243] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:49.758562 27901 raft_consensus.cc:2272] T 49241be91ca743cbac56eb24218ff775 P ca45420e033847489ad989c1c6eed3ef [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:50.166683 27901 tablet_server.cc:196] TabletServer@127.27.63.68:0 shutdown complete.
I20260709 13:13:50.229915 27901 master.cc:562] Master@127.27.63.126:43467 shutting down...
W20260709 13:13:50.267156 28956 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:13:50.267477 28956 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:13:50.267707 28956 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:13:53.733055 28892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111) [suppressed 381 similar messages]
W20260709 13:13:53.736850 28892 catalog_manager.cc:4755] TS 06ae4197fd784525b695a9b16c8094ea (127.27.63.65:43941): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2bbf1db4b075436695fd521f5a5b2e9d: Network error: Client connection negotiation failed: client connection to 127.27.63.65:43941: connect: Connection refused (error 111)
I20260709 13:13:53.883654 27901 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:53.884757 27901 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:53.885150 27901 tablet_replica.cc:333] T 00000000000000000000000000000000 P c44d1d5117734537b7202f74e0c1e920: stopping tablet replica
I20260709 13:13:54.078483 27901 master.cc:584] Master@127.27.63.126:43467 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (217755 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:13:54.193195 27901 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.63.126:36891
I20260709 13:13:54.194337 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:54.200107  3156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:54.200143  3157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:54.200959  3159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:54.201392 27901 server_base.cc:1061] running on GCE node
I20260709 13:13:54.203616 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:54.203784 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:54.203903 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602834203892 us; error 0 us; skew 500 ppm
I20260709 13:13:54.204412 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:54.206511 27901 webserver.cc:533] Webserver started at http://127.27.63.126:40481/ using document root <none> and password file <none>
I20260709 13:13:54.206952 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:54.207098 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:54.207304 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:54.208832 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/master-0-root/instance:
uuid: "48e1bd57ba224cbeb8fe214f140e07da"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.212874 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:13:54.215867  3164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.216581 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:13:54.216866 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/master-0-root
uuid: "48e1bd57ba224cbeb8fe214f140e07da"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.217132 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:54.243381 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:54.244643 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:54.282606 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.126:36891
I20260709 13:13:54.282737  3215 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.126:36891 every 8 connection(s)
I20260709 13:13:54.286641  3216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:54.297390  3216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da: Bootstrap starting.
I20260709 13:13:54.301994  3216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:54.305902  3216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da: No bootstrap required, opened a new log
I20260709 13:13:54.307971  3216 raft_consensus.cc:359] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e1bd57ba224cbeb8fe214f140e07da" member_type: VOTER }
I20260709 13:13:54.308467  3216 raft_consensus.cc:385] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:54.308643  3216 raft_consensus.cc:740] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e1bd57ba224cbeb8fe214f140e07da, State: Initialized, Role: FOLLOWER
I20260709 13:13:54.309206  3216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e1bd57ba224cbeb8fe214f140e07da" member_type: VOTER }
I20260709 13:13:54.309648  3216 raft_consensus.cc:399] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:13:54.309857  3216 raft_consensus.cc:493] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:13:54.310067  3216 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:54.314265  3216 raft_consensus.cc:515] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e1bd57ba224cbeb8fe214f140e07da" member_type: VOTER }
I20260709 13:13:54.314747  3216 leader_election.cc:304] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48e1bd57ba224cbeb8fe214f140e07da; no voters: 
I20260709 13:13:54.315773  3216 leader_election.cc:290] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:13:54.316233  3219 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:54.317579  3219 raft_consensus.cc:697] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 1 LEADER]: Becoming Leader. State: Replica: 48e1bd57ba224cbeb8fe214f140e07da, State: Running, Role: LEADER
I20260709 13:13:54.318176  3219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e1bd57ba224cbeb8fe214f140e07da" member_type: VOTER }
I20260709 13:13:54.318814  3216 sys_catalog.cc:565] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:13:54.325256  3220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48e1bd57ba224cbeb8fe214f140e07da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e1bd57ba224cbeb8fe214f140e07da" member_type: VOTER } }
I20260709 13:13:54.325996  3220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [sys.catalog]: This master's current role is: LEADER
I20260709 13:13:54.330674  3221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48e1bd57ba224cbeb8fe214f140e07da. Latest consensus state: current_term: 1 leader_uuid: "48e1bd57ba224cbeb8fe214f140e07da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e1bd57ba224cbeb8fe214f140e07da" member_type: VOTER } }
I20260709 13:13:54.331190  3221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [sys.catalog]: This master's current role is: LEADER
I20260709 13:13:54.332782  3227 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:13:54.337342  3227 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:13:54.343918 27901 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:13:54.346342  3227 catalog_manager.cc:1383] Generated new cluster ID: ba2f91071615424291be88d2dac35d34
I20260709 13:13:54.346622  3227 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:13:54.388638  3227 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:13:54.389919  3227 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:13:54.410063  3227 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da: Generated new TSK 0
I20260709 13:13:54.410629  3227 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:13:54.475289 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:54.481153  3237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:54.482851  3238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:54.485512  3240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:54.485896 27901 server_base.cc:1061] running on GCE node
I20260709 13:13:54.488099 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:54.488308 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:54.488437 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602834488426 us; error 0 us; skew 500 ppm
I20260709 13:13:54.488895 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:54.490939 27901 webserver.cc:533] Webserver started at http://127.27.63.65:35109/ using document root <none> and password file <none>
I20260709 13:13:54.491392 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:54.491544 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:54.491750 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:54.492955 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/instance:
uuid: "972a4469a4654416ab0a153d8c740282"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.497172 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:13:54.500612  3245 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.501341 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:13:54.501622 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "972a4469a4654416ab0a153d8c740282"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.501907 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:54.512953 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:54.514058 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:54.515439 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:54.518213 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:54.518421 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.518677 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:54.518867 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.559568 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:34091
I20260709 13:13:54.559661  3307 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:34091 every 8 connection(s)
I20260709 13:13:54.564795 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:54.572575  3312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:54.573403  3313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:54.576936  3315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:54.577486 27901 server_base.cc:1061] running on GCE node
I20260709 13:13:54.580104 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:54.580320 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:54.580508 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602834580474 us; error 0 us; skew 500 ppm
I20260709 13:13:54.581259 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:54.582976  3308 heartbeater.cc:344] Connected to a master server at 127.27.63.126:36891
I20260709 13:13:54.583323  3308 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:54.584009 27901 webserver.cc:533] Webserver started at http://127.27.63.66:45439/ using document root <none> and password file <none>
I20260709 13:13:54.584270  3308 heartbeater.cc:507] Master 127.27.63.126:36891 requested a full tablet report, sending...
I20260709 13:13:54.584636 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:54.584882 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:54.585137 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:54.586338 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/instance:
uuid: "83ef2126cf6d4f628cd668d0817871f2"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.586551  3181 ts_manager.cc:194] Registered new tserver with Master: 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:54.588512  3181 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50790
I20260709 13:13:54.591249 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:13:54.594668  3320 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.595445 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:13:54.595731 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-1-root
uuid: "83ef2126cf6d4f628cd668d0817871f2"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.596079 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:54.606163 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:54.607303 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:54.608747 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:54.610847 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:54.611069 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.611301 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:54.611475 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.652639 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.66:33623
I20260709 13:13:54.652760  3382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.66:33623 every 8 connection(s)
I20260709 13:13:54.657590 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:54.665064  3386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:54.669246  3387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:54.673349  3383 heartbeater.cc:344] Connected to a master server at 127.27.63.126:36891
I20260709 13:13:54.673601 27901 server_base.cc:1061] running on GCE node
I20260709 13:13:54.673811  3383 heartbeater.cc:461] Registering TS with master...
W20260709 13:13:54.674590  3389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:54.674654  3383 heartbeater.cc:507] Master 127.27.63.126:36891 requested a full tablet report, sending...
I20260709 13:13:54.675768 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:54.675982 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:54.676260 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602834676240 us; error 0 us; skew 500 ppm
I20260709 13:13:54.677047 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:54.677158  3181 ts_manager.cc:194] Registered new tserver with Master: 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:54.679080  3181 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50806
I20260709 13:13:54.679806 27901 webserver.cc:533] Webserver started at http://127.27.63.67:44223/ using document root <none> and password file <none>
I20260709 13:13:54.680454 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:54.680691 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:54.681022 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:54.682315 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/instance:
uuid: "06964934536148c2aa77a2f1c6382fea"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.686502 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:13:54.689836  3394 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.690615 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:13:54.690981 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-2-root
uuid: "06964934536148c2aa77a2f1c6382fea"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.691313 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:54.707489 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:54.708838 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:54.710572 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:54.712952 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:54.713246 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.713526 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:54.713742 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.755848 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.67:32907
I20260709 13:13:54.755949  3456 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.67:32907 every 8 connection(s)
I20260709 13:13:54.760860 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:54.768517  3461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:54.768929  3460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:54.772257 27901 server_base.cc:1061] running on GCE node
W20260709 13:13:54.773022  3463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:54.773558  3457 heartbeater.cc:344] Connected to a master server at 127.27.63.126:36891
I20260709 13:13:54.773928  3457 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:54.774646  3457 heartbeater.cc:507] Master 127.27.63.126:36891 requested a full tablet report, sending...
I20260709 13:13:54.775884 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:54.776226 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:54.776417 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602834776381 us; error 0 us; skew 500 ppm
I20260709 13:13:54.776762  3181 ts_manager.cc:194] Registered new tserver with Master: 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:54.777087 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:54.778297  3181 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50814
I20260709 13:13:54.779852 27901 webserver.cc:533] Webserver started at http://127.27.63.68:41993/ using document root <none> and password file <none>
I20260709 13:13:54.780400 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:54.780611 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:54.780884 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:54.782028 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-3-root/instance:
uuid: "73143d6f16c94a3aaef99f8e07588c1f"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.786432 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:13:54.789613  3468 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.790341 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:13:54.790647 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-3-root
uuid: "73143d6f16c94a3aaef99f8e07588c1f"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:13:54.790949 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:54.813862 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:54.815076 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:54.816375 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:54.818617 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:54.818849 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.819093 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:54.819267 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:54.869969 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.68:33961
I20260709 13:13:54.870074  3530 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.68:33961 every 8 connection(s)
I20260709 13:13:54.883198  3531 heartbeater.cc:344] Connected to a master server at 127.27.63.126:36891
I20260709 13:13:54.883587  3531 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:54.884392  3531 heartbeater.cc:507] Master 127.27.63.126:36891 requested a full tablet report, sending...
I20260709 13:13:54.886226  3181 ts_manager.cc:194] Registered new tserver with Master: 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:54.886723 27901 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.013427335s
I20260709 13:13:54.887578  3181 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50822
I20260709 13:13:54.912744  3181 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50832:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:13:55.175601  3422 tablet_service.cc:1511] Processing CreateTablet for tablet d1ff273fcfdc45a386458728eb7254f3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:13:55.177096  3422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ff273fcfdc45a386458728eb7254f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.178543  3496 tablet_service.cc:1511] Processing CreateTablet for tablet d1ff273fcfdc45a386458728eb7254f3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:13:55.180086  3496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ff273fcfdc45a386458728eb7254f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.181290  3421 tablet_service.cc:1511] Processing CreateTablet for tablet f4611ffabfbe4fa39f2fa67f79c3b048 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:13:55.182695  3421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4611ffabfbe4fa39f2fa67f79c3b048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.183076  3494 tablet_service.cc:1511] Processing CreateTablet for tablet d5cbb74eb76044f2a65c7a0b52c2adde (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:13:55.183841  3493 tablet_service.cc:1511] Processing CreateTablet for tablet b88e30e8276a41a98a3e4f6cf043375a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:13:55.185418  3495 tablet_service.cc:1511] Processing CreateTablet for tablet 94c6fcbbfd7244c0a66649594e57fb03 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:13:55.185818  3420 tablet_service.cc:1511] Processing CreateTablet for tablet d5cbb74eb76044f2a65c7a0b52c2adde (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:13:55.187228  3420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cbb74eb76044f2a65c7a0b52c2adde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.187395  3494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cbb74eb76044f2a65c7a0b52c2adde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.192828  3493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88e30e8276a41a98a3e4f6cf043375a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.197490  3495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c6fcbbfd7244c0a66649594e57fb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.206624  3419 tablet_service.cc:1511] Processing CreateTablet for tablet 3061a90b8dbc42488cec94032788e597 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:13:55.208590  3348 tablet_service.cc:1511] Processing CreateTablet for tablet f4611ffabfbe4fa39f2fa67f79c3b048 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:13:55.209246  3418 tablet_service.cc:1511] Processing CreateTablet for tablet 78ac2072e92b45fd89cb3b4057fb9e7b (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:13:55.210005  3348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4611ffabfbe4fa39f2fa67f79c3b048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.211331  3272 tablet_service.cc:1511] Processing CreateTablet for tablet f4611ffabfbe4fa39f2fa67f79c3b048 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:13:55.211630  3417 tablet_service.cc:1511] Processing CreateTablet for tablet 514a52a2733c4c2f996475e12c2dbaa8 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:13:55.212721  3272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4611ffabfbe4fa39f2fa67f79c3b048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.212821  3270 tablet_service.cc:1511] Processing CreateTablet for tablet 3061a90b8dbc42488cec94032788e597 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:13:55.214294  3270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3061a90b8dbc42488cec94032788e597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.214059  3416 tablet_service.cc:1511] Processing CreateTablet for tablet 018917aa98ef46b4990e809f93148c34 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:13:55.216329  3415 tablet_service.cc:1511] Processing CreateTablet for tablet c4e38e5f24604ea48097687a200d7e48 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:13:55.217260  3346 tablet_service.cc:1511] Processing CreateTablet for tablet d5cbb74eb76044f2a65c7a0b52c2adde (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:13:55.218568  3346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cbb74eb76044f2a65c7a0b52c2adde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.218678  3414 tablet_service.cc:1511] Processing CreateTablet for tablet 691ec311298b4d19a1e62763ea313587 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:13:55.221243  3345 tablet_service.cc:1511] Processing CreateTablet for tablet 3061a90b8dbc42488cec94032788e597 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:13:55.221514  3273 tablet_service.cc:1511] Processing CreateTablet for tablet d1ff273fcfdc45a386458728eb7254f3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:13:55.222645  3345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3061a90b8dbc42488cec94032788e597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.222723  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ff273fcfdc45a386458728eb7254f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.225003  3339 tablet_service.cc:1511] Processing CreateTablet for tablet f058e8e6fdfa480db59f5004728b8f80 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:13:55.225673  3344 tablet_service.cc:1511] Processing CreateTablet for tablet b88e30e8276a41a98a3e4f6cf043375a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:13:55.226354  3339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f058e8e6fdfa480db59f5004728b8f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.227051  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88e30e8276a41a98a3e4f6cf043375a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.228343  3347 tablet_service.cc:1511] Processing CreateTablet for tablet 94c6fcbbfd7244c0a66649594e57fb03 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:13:55.228550  3269 tablet_service.cc:1511] Processing CreateTablet for tablet b88e30e8276a41a98a3e4f6cf043375a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:13:55.229713  3347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c6fcbbfd7244c0a66649594e57fb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.231249  3268 tablet_service.cc:1511] Processing CreateTablet for tablet 78ac2072e92b45fd89cb3b4057fb9e7b (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:13:55.229744  3269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88e30e8276a41a98a3e4f6cf043375a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.232666  3268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78ac2072e92b45fd89cb3b4057fb9e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.237617  3341 tablet_service.cc:1511] Processing CreateTablet for tablet 5d04edf84bba4d759de41bab8b424463 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:13:55.211632  3271 tablet_service.cc:1511] Processing CreateTablet for tablet 94c6fcbbfd7244c0a66649594e57fb03 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:13:55.247363  3489 tablet_service.cc:1511] Processing CreateTablet for tablet c4e38e5f24604ea48097687a200d7e48 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:13:55.240713  3413 tablet_service.cc:1511] Processing CreateTablet for tablet f058e8e6fdfa480db59f5004728b8f80 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:13:55.248749  3489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e38e5f24604ea48097687a200d7e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.249454  3341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d04edf84bba4d759de41bab8b424463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.240713  3264 tablet_service.cc:1511] Processing CreateTablet for tablet f058e8e6fdfa480db59f5004728b8f80 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:13:55.241850  3342 tablet_service.cc:1511] Processing CreateTablet for tablet 018917aa98ef46b4990e809f93148c34 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:13:55.254904  3342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 018917aa98ef46b4990e809f93148c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.255410  3422 tablet_service.cc:1511] Processing CreateTablet for tablet d7574b0cc43f4903a703314b82b0e9da (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:13:55.242997  3340 tablet_service.cc:1511] Processing CreateTablet for tablet 691ec311298b4d19a1e62763ea313587 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:13:55.257323  3340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691ec311298b4d19a1e62763ea313587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.244064  3491 tablet_service.cc:1511] Processing CreateTablet for tablet 514a52a2733c4c2f996475e12c2dbaa8 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:13:55.261400  3491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514a52a2733c4c2f996475e12c2dbaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.245159  3265 tablet_service.cc:1511] Processing CreateTablet for tablet c4e38e5f24604ea48097687a200d7e48 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:13:55.245858  3266 tablet_service.cc:1511] Processing CreateTablet for tablet 5d04edf84bba4d759de41bab8b424463 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:13:55.246444  3343 tablet_service.cc:1511] Processing CreateTablet for tablet 514a52a2733c4c2f996475e12c2dbaa8 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:13:55.266834  3343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514a52a2733c4c2f996475e12c2dbaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.246598  3267 tablet_service.cc:1511] Processing CreateTablet for tablet 018917aa98ef46b4990e809f93148c34 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:13:55.242373  3492 tablet_service.cc:1511] Processing CreateTablet for tablet 78ac2072e92b45fd89cb3b4057fb9e7b (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:13:55.270628  3492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78ac2072e92b45fd89cb3b4057fb9e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.245659  3490 tablet_service.cc:1511] Processing CreateTablet for tablet 5d04edf84bba4d759de41bab8b424463 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:13:55.273473  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d04edf84bba4d759de41bab8b424463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.279176  3552 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:55.279945  3271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c6fcbbfd7244c0a66649594e57fb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.283629  3419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3061a90b8dbc42488cec94032788e597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.283629  3266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d04edf84bba4d759de41bab8b424463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.288362  3264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f058e8e6fdfa480db59f5004728b8f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.296203  3415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e38e5f24604ea48097687a200d7e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.296898  3418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78ac2072e92b45fd89cb3b4057fb9e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.297597  3413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f058e8e6fdfa480db59f5004728b8f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.298307  3414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691ec311298b4d19a1e62763ea313587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.299537  3416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 018917aa98ef46b4990e809f93148c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.300838  3422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7574b0cc43f4903a703314b82b0e9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.302984  3417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514a52a2733c4c2f996475e12c2dbaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.307294  3488 tablet_service.cc:1511] Processing CreateTablet for tablet 691ec311298b4d19a1e62763ea313587 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:13:55.307505  3487 tablet_service.cc:1511] Processing CreateTablet for tablet a468aba2a1664f7b8a5e1755775a2e72 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:13:55.308878  3487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a468aba2a1664f7b8a5e1755775a2e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.313349  3488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691ec311298b4d19a1e62763ea313587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.329613  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 018917aa98ef46b4990e809f93148c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.313371  3272 tablet_service.cc:1511] Processing CreateTablet for tablet a468aba2a1664f7b8a5e1755775a2e72 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:13:55.337445  3272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a468aba2a1664f7b8a5e1755775a2e72. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:55.339279  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:55.346994  3420 tablet_service.cc:1511] Processing CreateTablet for tablet beb78494b2e0410ba8b06a30fff7158a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:13:55.348564  3420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb78494b2e0410ba8b06a30fff7158a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.359469  3551 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:55.364480  3265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e38e5f24604ea48097687a200d7e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.393925  3421 tablet_service.cc:1511] Processing CreateTablet for tablet bf2d92deaf5c4766a2cc54da4c274434 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:13:55.395354  3421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf2d92deaf5c4766a2cc54da4c274434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.402339  3552 tablet_bootstrap.cc:654] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.429844  3551 tablet_bootstrap.cc:654] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.435189  3495 tablet_service.cc:1511] Processing CreateTablet for tablet d7574b0cc43f4903a703314b82b0e9da (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:13:55.436424  3347 tablet_service.cc:1511] Processing CreateTablet for tablet a468aba2a1664f7b8a5e1755775a2e72 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:13:55.437121  3271 tablet_service.cc:1511] Processing CreateTablet for tablet d7574b0cc43f4903a703314b82b0e9da (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:13:55.438848  3267 tablet_service.cc:1511] Processing CreateTablet for tablet bf2d92deaf5c4766a2cc54da4c274434 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:13:55.440147  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf2d92deaf5c4766a2cc54da4c274434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.441329  3346 tablet_service.cc:1511] Processing CreateTablet for tablet beb78494b2e0410ba8b06a30fff7158a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:13:55.447871  3271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7574b0cc43f4903a703314b82b0e9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.449141  3413 tablet_service.cc:1511] Processing CreateTablet for tablet 9042f5a22d47425eac26b6f03be99d9d (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:13:55.450556  3413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9042f5a22d47425eac26b6f03be99d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.456889  3495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7574b0cc43f4903a703314b82b0e9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.466090  3273 tablet_service.cc:1511] Processing CreateTablet for tablet 46e6844fd8644c6498d5b9ce0e7cbd6b (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:13:55.477092  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e6844fd8644c6498d5b9ce0e7cbd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.478122  3496 tablet_service.cc:1511] Processing CreateTablet for tablet beb78494b2e0410ba8b06a30fff7158a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:13:55.479535  3496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb78494b2e0410ba8b06a30fff7158a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.484072  3488 tablet_service.cc:1511] Processing CreateTablet for tablet 46e6844fd8644c6498d5b9ce0e7cbd6b (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:13:55.437704  3347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a468aba2a1664f7b8a5e1755775a2e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.501178  3488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e6844fd8644c6498d5b9ce0e7cbd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.520200  3346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb78494b2e0410ba8b06a30fff7158a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.521095  3492 tablet_service.cc:1511] Processing CreateTablet for tablet 9042f5a22d47425eac26b6f03be99d9d (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:13:55.522415  3492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9042f5a22d47425eac26b6f03be99d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.526242  3554 tablet_bootstrap.cc:492] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:55.533249  3343 tablet_service.cc:1511] Processing CreateTablet for tablet bf2d92deaf5c4766a2cc54da4c274434 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:13:55.534284  3343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf2d92deaf5c4766a2cc54da4c274434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.536471  3554 tablet_bootstrap.cc:654] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.541029  3270 tablet_service.cc:1511] Processing CreateTablet for tablet 9042f5a22d47425eac26b6f03be99d9d (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:13:55.542484  3270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9042f5a22d47425eac26b6f03be99d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.544137  3553 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:55.576534  3344 tablet_service.cc:1511] Processing CreateTablet for tablet 46e6844fd8644c6498d5b9ce0e7cbd6b (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:13:55.577955  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e6844fd8644c6498d5b9ce0e7cbd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.581305  3552 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:55.581920  3552 ts_tablet_manager.cc:1403] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.303s	user 0.018s	sys 0.006s
I20260709 13:13:55.585057  3552 raft_consensus.cc:359] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:55.585841  3552 raft_consensus.cc:385] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:55.586210  3552 raft_consensus.cc:740] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:55.587066  3552 consensus_queue.cc:260] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:55.591382  3308 heartbeater.cc:499] Master 127.27.63.126:36891 was elected leader, sending a full tablet report...
I20260709 13:13:55.593804  3551 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:55.594197  3551 ts_tablet_manager.cc:1403] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.235s	user 0.018s	sys 0.014s
I20260709 13:13:55.596616  3551 raft_consensus.cc:359] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:55.596722  3340 tablet_service.cc:1511] Processing CreateTablet for tablet f23621b920b44c8dbdd783bed09dfa50 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:13:55.597364  3551 raft_consensus.cc:385] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:55.597766  3551 raft_consensus.cc:740] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:55.598148  3340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f23621b920b44c8dbdd783bed09dfa50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.598709  3551 consensus_queue.cc:260] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:55.609241  3413 tablet_service.cc:1511] Processing CreateTablet for tablet f23621b920b44c8dbdd783bed09dfa50 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:13:55.611038  3413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f23621b920b44c8dbdd783bed09dfa50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.614903  3494 tablet_service.cc:1511] Processing CreateTablet for tablet f23621b920b44c8dbdd783bed09dfa50 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:13:55.615569  3272 tablet_service.cc:1511] Processing CreateTablet for tablet d732d672e75142f79a290bd7f5ce0840 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:13:55.616688  3494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f23621b920b44c8dbdd783bed09dfa50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.616866  3272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d732d672e75142f79a290bd7f5ce0840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.626834  3348 tablet_service.cc:1511] Processing CreateTablet for tablet d732d672e75142f79a290bd7f5ce0840 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:13:55.628250  3348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d732d672e75142f79a290bd7f5ce0840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.638051  3416 tablet_service.cc:1511] Processing CreateTablet for tablet d732d672e75142f79a290bd7f5ce0840 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:13:55.639415  3416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d732d672e75142f79a290bd7f5ce0840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.646759  3490 tablet_service.cc:1511] Processing CreateTablet for tablet 98065e64beff449b8caef22a3f2a49f5 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:13:55.648162  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98065e64beff449b8caef22a3f2a49f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.648514  3553 tablet_bootstrap.cc:654] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.657115  3267 tablet_service.cc:1511] Processing CreateTablet for tablet 98065e64beff449b8caef22a3f2a49f5 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:13:55.658851  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98065e64beff449b8caef22a3f2a49f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.690085  3554 tablet_bootstrap.cc:492] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:55.689850  3421 tablet_service.cc:1511] Processing CreateTablet for tablet 87e493d066c44520bd444d8afc63ec91 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:13:55.690582  3554 ts_tablet_manager.cc:1403] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.165s	user 0.015s	sys 0.039s
I20260709 13:13:55.690809  3551 ts_tablet_manager.cc:1434] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.096s	user 0.032s	sys 0.020s
I20260709 13:13:55.691187  3421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e493d066c44520bd444d8afc63ec91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.691819  3551 tablet_bootstrap.cc:492] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:55.683601  3457 heartbeater.cc:499] Master 127.27.63.126:36891 was elected leader, sending a full tablet report...
I20260709 13:13:55.695665  3273 tablet_service.cc:1511] Processing CreateTablet for tablet 87e493d066c44520bd444d8afc63ec91 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:13:55.698767  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e493d066c44520bd444d8afc63ec91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.698622  3264 tablet_service.cc:1511] Processing CreateTablet for tablet 208caff3bfdf46a495a86f06a667659a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:13:55.699466  3493 tablet_service.cc:1511] Processing CreateTablet for tablet 65d7199324524f1fbb9a40b53925a3a1 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:13:55.699894  3264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208caff3bfdf46a495a86f06a667659a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.700734  3493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65d7199324524f1fbb9a40b53925a3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.687408  3383 heartbeater.cc:499] Master 127.27.63.126:36891 was elected leader, sending a full tablet report...
I20260709 13:13:55.710721  3419 tablet_service.cc:1511] Processing CreateTablet for tablet 65d7199324524f1fbb9a40b53925a3a1 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:13:55.693161  3552 ts_tablet_manager.cc:1434] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.111s	user 0.012s	sys 0.015s
I20260709 13:13:55.719560  3553 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:55.712006  3419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65d7199324524f1fbb9a40b53925a3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.720098  3553 ts_tablet_manager.cc:1403] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.176s	user 0.017s	sys 0.016s
I20260709 13:13:55.721302  3551 tablet_bootstrap.cc:654] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.721110  3346 tablet_service.cc:1511] Processing CreateTablet for tablet 65d7199324524f1fbb9a40b53925a3a1 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:13:55.722414  3346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65d7199324524f1fbb9a40b53925a3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.723124  3553 raft_consensus.cc:359] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:55.723836  3553 raft_consensus.cc:385] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:55.724145  3553 raft_consensus.cc:740] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:55.725036  3553 consensus_queue.cc:260] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:55.731889  3552 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:55.736172  3344 tablet_service.cc:1511] Processing CreateTablet for tablet 98065e64beff449b8caef22a3f2a49f5 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:13:55.738075  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98065e64beff449b8caef22a3f2a49f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.739634  3552 tablet_bootstrap.cc:654] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.750418  3347 tablet_service.cc:1511] Processing CreateTablet for tablet 208caff3bfdf46a495a86f06a667659a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:13:55.751995  3347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208caff3bfdf46a495a86f06a667659a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:55.764231  3458 tablet.cc:2406] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:13:55.764709  3418 tablet_service.cc:1511] Processing CreateTablet for tablet 208caff3bfdf46a495a86f06a667659a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:13:55.766449  3418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208caff3bfdf46a495a86f06a667659a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.770040  3494 tablet_service.cc:1511] Processing CreateTablet for tablet 5557da3aee9b41f793515bea5f1f42cc (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:13:55.771461  3494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5557da3aee9b41f793515bea5f1f42cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.776935  3554 raft_consensus.cc:359] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:55.777959  3554 raft_consensus.cc:385] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:55.778321  3554 raft_consensus.cc:740] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:55.782353  3270 tablet_service.cc:1511] Processing CreateTablet for tablet 5557da3aee9b41f793515bea5f1f42cc (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:13:55.783612  3270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5557da3aee9b41f793515bea5f1f42cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.750872  3496 tablet_service.cc:1511] Processing CreateTablet for tablet 87e493d066c44520bd444d8afc63ec91 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:13:55.784011  3347 tablet_service.cc:1511] Processing CreateTablet for tablet 5557da3aee9b41f793515bea5f1f42cc (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:13:55.784904  3496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e493d066c44520bd444d8afc63ec91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.784675  3414 tablet_service.cc:1511] Processing CreateTablet for tablet 361bb4036c344c3cbf83650eeaf091aa (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:13:55.785275  3347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5557da3aee9b41f793515bea5f1f42cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.785916  3414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361bb4036c344c3cbf83650eeaf091aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.779253  3554 consensus_queue.cc:260] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:55.796021  3492 tablet_service.cc:1511] Processing CreateTablet for tablet 361bb4036c344c3cbf83650eeaf091aa (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:13:55.797441  3492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361bb4036c344c3cbf83650eeaf091aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.802304  3553 ts_tablet_manager.cc:1434] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.082s	user 0.023s	sys 0.004s
I20260709 13:13:55.803383  3553 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:55.813650  3553 tablet_bootstrap.cc:654] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.822746  3554 ts_tablet_manager.cc:1434] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.132s	user 0.012s	sys 0.000s
I20260709 13:13:55.823987  3554 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:55.830957  3554 tablet_bootstrap.cc:654] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.836565  3558 raft_consensus.cc:493] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.837113  3558 raft_consensus.cc:515] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:55.839603  3558 leader_election.cc:290] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:55.840876  3531 heartbeater.cc:499] Master 127.27.63.126:36891 was elected leader, sending a full tablet report...
W20260709 13:13:55.873189  3167 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 56122us
I20260709 13:13:55.887085  3270 tablet_service.cc:1511] Processing CreateTablet for tablet 361bb4036c344c3cbf83650eeaf091aa (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:13:55.888834  3270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361bb4036c344c3cbf83650eeaf091aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.898586  3343 tablet_service.cc:1511] Processing CreateTablet for tablet c49acd4ec7dc42a8b9a1cb2835d1bede (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:13:55.899950  3343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49acd4ec7dc42a8b9a1cb2835d1bede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.904487  3493 tablet_service.cc:1511] Processing CreateTablet for tablet c49acd4ec7dc42a8b9a1cb2835d1bede (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:13:55.906522  3413 tablet_service.cc:1511] Processing CreateTablet for tablet c49acd4ec7dc42a8b9a1cb2835d1bede (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:13:55.907549  3493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49acd4ec7dc42a8b9a1cb2835d1bede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.907799  3413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49acd4ec7dc42a8b9a1cb2835d1bede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.906911  3551 tablet_bootstrap.cc:492] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:55.912653  3551 ts_tablet_manager.cc:1403] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.221s	user 0.025s	sys 0.033s
I20260709 13:13:55.923270  3552 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:55.924005  3552 ts_tablet_manager.cc:1403] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.192s	user 0.027s	sys 0.030s
I20260709 13:13:55.929068  3551 raft_consensus.cc:359] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:55.930068  3551 raft_consensus.cc:385] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:55.930451  3551 raft_consensus.cc:740] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:55.931607  3551 consensus_queue.cc:260] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:55.941795  3551 ts_tablet_manager.cc:1434] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.002s
I20260709 13:13:55.943118  3551 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:55.959808  3554 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:55.960609  3554 ts_tablet_manager.cc:1403] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.137s	user 0.012s	sys 0.009s
I20260709 13:13:55.953224  3552 raft_consensus.cc:359] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:55.961951  3552 raft_consensus.cc:385] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:55.964597  3552 raft_consensus.cc:740] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:55.971076  3490 tablet_service.cc:1511] Processing CreateTablet for tablet 59159d79fbfc4835a64e52a2783f46f7 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:13:55.971454  3264 tablet_service.cc:1511] Processing CreateTablet for tablet 59159d79fbfc4835a64e52a2783f46f7 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:13:55.972810  3264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59159d79fbfc4835a64e52a2783f46f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.972885  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59159d79fbfc4835a64e52a2783f46f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.977029  3340 tablet_service.cc:1511] Processing CreateTablet for tablet 59159d79fbfc4835a64e52a2783f46f7 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:13:55.978363  3340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59159d79fbfc4835a64e52a2783f46f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.979450  3346 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6db7a8007940d6b8bc088af66589ae (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:13:55.979252  3552 consensus_queue.cc:260] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:55.980728  3346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6db7a8007940d6b8bc088af66589ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:55.987674  3551 tablet_bootstrap.cc:654] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:55.994482  3553 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:55.995013  3553 ts_tablet_manager.cc:1403] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.192s	user 0.026s	sys 0.023s
I20260709 13:13:55.998183  3553 raft_consensus.cc:359] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:55.998893  3553 raft_consensus.cc:385] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.006084  3270 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6db7a8007940d6b8bc088af66589ae (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:13:56.007431  3270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6db7a8007940d6b8bc088af66589ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.009922  3554 raft_consensus.cc:359] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.010720  3554 raft_consensus.cc:385] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.011026  3554 raft_consensus.cc:740] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.025622  3490 tablet_service.cc:1511] Processing CreateTablet for tablet bb18f85836cc456b8362bb08424827ca (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:13:56.025880  3273 tablet_service.cc:1511] Processing CreateTablet for tablet bb18f85836cc456b8362bb08424827ca (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:13:56.027031  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb18f85836cc456b8362bb08424827ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.026739  3348 tablet_service.cc:1511] Processing CreateTablet for tablet 8051a5cc00534c788d643e0b21d3d9c2 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:13:56.028100  3348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8051a5cc00534c788d643e0b21d3d9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.011761  3554 consensus_queue.cc:260] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.037129  3552 ts_tablet_manager.cc:1434] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.113s	user 0.009s	sys 0.000s
I20260709 13:13:56.027033  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb18f85836cc456b8362bb08424827ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.038359  3552 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:56.041783  3554 ts_tablet_manager.cc:1434] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.081s	user 0.004s	sys 0.002s
I20260709 13:13:56.042820  3554 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:56.046746  3493 tablet_service.cc:1511] Processing CreateTablet for tablet 8051a5cc00534c788d643e0b21d3d9c2 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:13:56.047513  3558 raft_consensus.cc:493] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.048146  3493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8051a5cc00534c788d643e0b21d3d9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.048013  3558 raft_consensus.cc:515] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:55.999184  3553 raft_consensus.cc:740] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.048913  3343 tablet_service.cc:1511] Processing CreateTablet for tablet b61ff345dd8d4467829eded70f16f6a0 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:13:56.049428  3553 consensus_queue.cc:260] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.050066  3558 leader_election.cc:290] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:56.050230  3343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61ff345dd8d4467829eded70f16f6a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.051941  3553 ts_tablet_manager.cc:1434] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.001s
I20260709 13:13:56.052950  3553 tablet_bootstrap.cc:492] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:56.063133  3264 tablet_service.cc:1511] Processing CreateTablet for tablet b61ff345dd8d4467829eded70f16f6a0 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:13:56.064536  3264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61ff345dd8d4467829eded70f16f6a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.068464  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:56.073495  3494 tablet_service.cc:1511] Processing CreateTablet for tablet f7e3b18f25574f46ac56a124bb52c1c3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:13:56.074360  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "018917aa98ef46b4990e809f93148c34" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:56.074923  3494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e3b18f25574f46ac56a124bb52c1c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.076208  3413 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6db7a8007940d6b8bc088af66589ae (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:13:56.074951  3418 tablet_service.cc:1511] Processing CreateTablet for tablet bb18f85836cc456b8362bb08424827ca (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:13:56.076458  3419 tablet_service.cc:1511] Processing CreateTablet for tablet 8051a5cc00534c788d643e0b21d3d9c2 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:13:56.077235  3559 raft_consensus.cc:493] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.077538  3413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6db7a8007940d6b8bc088af66589ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.077740  3559 raft_consensus.cc:515] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.078599  3418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb18f85836cc456b8362bb08424827ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.079876  3416 tablet_service.cc:1511] Processing CreateTablet for tablet b61ff345dd8d4467829eded70f16f6a0 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:13:56.080528  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:56.084259  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "018917aa98ef46b4990e809f93148c34" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:56.086181  3432 raft_consensus.cc:2468] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:56.086264  3272 tablet_service.cc:1511] Processing CreateTablet for tablet f7e3b18f25574f46ac56a124bb52c1c3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:13:56.087370  3559 leader_election.cc:290] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:56.087677  3272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e3b18f25574f46ac56a124bb52c1c3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:56.077564  3247 leader_election.cc:343] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.104346  3554 tablet_bootstrap.cc:654] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.104614  3551 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:56.104401  3344 tablet_service.cc:1511] Processing CreateTablet for tablet f7e3b18f25574f46ac56a124bb52c1c3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:13:56.105238  3551 ts_tablet_manager.cc:1403] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.162s	user 0.008s	sys 0.006s
I20260709 13:13:56.105953  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e3b18f25574f46ac56a124bb52c1c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.115417  3421 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1d609ecf684c38b48d776083ce2417 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:13:56.116035  3492 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1d609ecf684c38b48d776083ce2417 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:13:56.115391  3419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8051a5cc00534c788d643e0b21d3d9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.117448  3492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1d609ecf684c38b48d776083ce2417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.120338  3552 tablet_bootstrap.cc:654] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.121155  3551 raft_consensus.cc:359] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.121917  3551 raft_consensus.cc:385] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.122160  3416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61ff345dd8d4467829eded70f16f6a0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:56.103087  3247 leader_election.cc:343] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.122200  3551 raft_consensus.cc:740] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.128552  3247 leader_election.cc:304] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:56.128798  3551 consensus_queue.cc:260] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.129419  3558 raft_consensus.cc:2804] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:56.129781  3558 raft_consensus.cc:493] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.130112  3558 raft_consensus.cc:3060] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.133352  3551 ts_tablet_manager.cc:1434] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.003s
I20260709 13:13:56.139061  3551 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
W20260709 13:13:56.146368  3247 leader_election.cc:343] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.153614  3421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1d609ecf684c38b48d776083ce2417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.151542  3273 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1d609ecf684c38b48d776083ce2417 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:13:56.152801  3247 leader_election.cc:304] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972a4469a4654416ab0a153d8c740282; no voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:56.155202  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1d609ecf684c38b48d776083ce2417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.200147  3576 raft_consensus.cc:2749] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:56.201889  3551 tablet_bootstrap.cc:654] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.203328  3562 raft_consensus.cc:493] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.203847  3562 raft_consensus.cc:515] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:56.206514  3562 leader_election.cc:290] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:56.208664  3562 raft_consensus.cc:493] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.209239  3562 raft_consensus.cc:515] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.210541  3553 tablet_bootstrap.cc:654] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.212298  3564 raft_consensus.cc:493] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.212946  3564 raft_consensus.cc:515] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.211844  3562 leader_election.cc:290] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:56.231499  3552 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:56.232239  3552 ts_tablet_manager.cc:1403] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.194s	user 0.013s	sys 0.004s
I20260709 13:13:56.236425  3552 raft_consensus.cc:359] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.237118  3552 raft_consensus.cc:385] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.237563  3552 raft_consensus.cc:740] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.223927  3564 leader_election.cc:290] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
W20260709 13:13:56.244246  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 97904.5us
I20260709 13:13:56.257239  3564 raft_consensus.cc:493] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.268136  3564 raft_consensus.cc:515] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:56.270509  3564 leader_election.cc:290] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:56.270962  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:56.265561  3552 consensus_queue.cc:260] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.281333  3552 ts_tablet_manager.cc:1434] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.002s
I20260709 13:13:56.284931  3267 tablet_service.cc:1511] Processing CreateTablet for tablet 304d003e5a044246b2968b09bafc2146 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:13:56.285379  3414 tablet_service.cc:1511] Processing CreateTablet for tablet 596e0222214f40b5861691b7bcaeea24 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:13:56.286715  3414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596e0222214f40b5861691b7bcaeea24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.293954  3493 tablet_service.cc:1511] Processing CreateTablet for tablet 596e0222214f40b5861691b7bcaeea24 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:13:56.294046  3496 tablet_service.cc:1511] Processing CreateTablet for tablet 094d0615636b4da4a2c8ba92b4218624 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:13:56.295373  3493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596e0222214f40b5861691b7bcaeea24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.295380  3420 tablet_service.cc:1511] Processing CreateTablet for tablet 304d003e5a044246b2968b09bafc2146 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:13:56.263065  3558 raft_consensus.cc:515] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:13:56.315470  3396 leader_election.cc:343] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.325125  3271 tablet_service.cc:1511] Processing CreateTablet for tablet 094d0615636b4da4a2c8ba92b4218624 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:13:56.326589  3271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094d0615636b4da4a2c8ba92b4218624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.313738  3496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094d0615636b4da4a2c8ba92b4218624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.331193  3558 leader_election.cc:290] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:56.332320  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:56.332831  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:56.336253  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304d003e5a044246b2968b09bafc2146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.340854  3432 raft_consensus.cc:3060] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:56.342422  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:13:56.343394  3247 leader_election.cc:343] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.344470  3552 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:56.339836  3420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304d003e5a044246b2968b09bafc2146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.349365  3554 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:56.349900  3554 ts_tablet_manager.cc:1403] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.307s	user 0.017s	sys 0.017s
I20260709 13:13:56.356259  3490 tablet_service.cc:1511] Processing CreateTablet for tablet a1128833b31748d0b098825fd4212f82 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:13:56.357138  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:56.357640  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1128833b31748d0b098825fd4212f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.358048  3283 raft_consensus.cc:2393] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06964934536148c2aa77a2f1c6382fea in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:13:56.358408  3432 raft_consensus.cc:2468] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:56.359463  3397 leader_election.cc:304] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:56.359650  3247 leader_election.cc:304] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:56.360574  3558 raft_consensus.cc:2804] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:56.360867  3559 raft_consensus.cc:2749] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:56.361030  3558 raft_consensus.cc:697] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:56.362085  3558 consensus_queue.cc:237] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.362938  3559 raft_consensus.cc:493] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.363387  3559 raft_consensus.cc:515] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:56.365728  3559 leader_election.cc:290] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:56.427381  3551 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:56.427229  3344 tablet_service.cc:1511] Processing CreateTablet for tablet 596e0222214f40b5861691b7bcaeea24 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:13:56.427923  3551 ts_tablet_manager.cc:1403] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.289s	user 0.015s	sys 0.012s
I20260709 13:13:56.428592  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596e0222214f40b5861691b7bcaeea24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.431077  3551 raft_consensus.cc:359] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.432940  3419 tablet_service.cc:1511] Processing CreateTablet for tablet a1128833b31748d0b098825fd4212f82 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:13:56.434208  3419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1128833b31748d0b098825fd4212f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.418246  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cbb74eb76044f2a65c7a0b52c2adde" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:13:56.440426  3396 leader_election.cc:343] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.431828  3551 raft_consensus.cc:385] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.446250  3552 tablet_bootstrap.cc:654] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.450094  3492 tablet_service.cc:1511] Processing CreateTablet for tablet e1bb6275bf5a44debe9b7cdd0d757a3e (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:13:56.456511  3273 tablet_service.cc:1511] Processing CreateTablet for tablet a1128833b31748d0b098825fd4212f82 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:13:56.457891  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1128833b31748d0b098825fd4212f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.455917  3559 raft_consensus.cc:493] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.459515  3559 raft_consensus.cc:515] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.458925  3178 catalog_manager.cc:5697] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:56.461575  3559 leader_election.cc:290] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:56.461889  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:56.462646  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:56.462826  3554 raft_consensus.cc:359] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.463351  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c6fcbbfd7244c0a66649594e57fb03" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:56.463589  3554 raft_consensus.cc:385] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.463914  3554 raft_consensus.cc:740] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.463995  3505 raft_consensus.cc:2468] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
W20260709 13:13:56.464094  3397 leader_election.cc:343] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.464713  3554 consensus_queue.cc:260] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:13:56.465740  3324 leader_election.cc:343] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.466454  3324 leader_election.cc:304] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:56.467602  3562 raft_consensus.cc:2804] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:56.468014  3562 raft_consensus.cc:493] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.468365  3562 raft_consensus.cc:3060] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.469019  3414 tablet_service.cc:1511] Processing CreateTablet for tablet e1bb6275bf5a44debe9b7cdd0d757a3e (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:13:56.470325  3414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1bb6275bf5a44debe9b7cdd0d757a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.467058  3554 ts_tablet_manager.cc:1434] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.117s	user 0.007s	sys 0.000s
I20260709 13:13:56.478067  3562 raft_consensus.cc:515] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.479242  3492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1bb6275bf5a44debe9b7cdd0d757a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.445611  3551 raft_consensus.cc:740] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.493261  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cbb74eb76044f2a65c7a0b52c2adde" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:56.494035  3432 raft_consensus.cc:2468] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:56.493316  3343 tablet_service.cc:1511] Processing CreateTablet for tablet 304d003e5a044246b2968b09bafc2146 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:13:56.499082  3562 leader_election.cc:290] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:56.499349  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c6fcbbfd7244c0a66649594e57fb03" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:56.504077  3505 raft_consensus.cc:3060] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.504609  3348 tablet_service.cc:1511] Processing CreateTablet for tablet 094d0615636b4da4a2c8ba92b4218624 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:13:56.500455  3343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304d003e5a044246b2968b09bafc2146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.508595  3348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094d0615636b4da4a2c8ba92b4218624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.520141  3554 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:56.526364  3413 tablet_service.cc:1511] Processing CreateTablet for tablet e514dae1f1814a2b80b772b07399cc58 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:13:56.526695  3470 leader_election.cc:304] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:13:56.527734  3413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e514dae1f1814a2b80b772b07399cc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.512889  3551 consensus_queue.cc:260] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.531327  3552 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:56.531826  3552 ts_tablet_manager.cc:1403] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.188s	user 0.005s	sys 0.020s
I20260709 13:13:56.533557  3551 ts_tablet_manager.cc:1434] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.105s	user 0.006s	sys 0.000s
I20260709 13:13:56.535004  3551 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:56.534691  3552 raft_consensus.cc:359] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.535459  3552 raft_consensus.cc:385] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.532101  3493 tablet_service.cc:1511] Processing CreateTablet for tablet f9ccb9fa24984b6c9a11b8429cd45338 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:13:56.527746  3584 raft_consensus.cc:2804] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:56.536561  3493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9ccb9fa24984b6c9a11b8429cd45338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.536597  3584 raft_consensus.cc:493] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.537155  3584 raft_consensus.cc:3060] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.535763  3552 raft_consensus.cc:740] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.540490  3490 tablet_service.cc:1511] Processing CreateTablet for tablet 7c16129126724cb3a6e2ecfa3547b4e6 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:13:56.540540  3273 tablet_service.cc:1511] Processing CreateTablet for tablet e514dae1f1814a2b80b772b07399cc58 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:13:56.542588  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e514dae1f1814a2b80b772b07399cc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.543334  3553 tablet_bootstrap.cc:492] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:56.543843  3553 ts_tablet_manager.cc:1403] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.491s	user 0.014s	sys 0.021s
I20260709 13:13:56.548243  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cbb74eb76044f2a65c7a0b52c2adde" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:56.549777  3346 tablet_service.cc:1511] Processing CreateTablet for tablet e514dae1f1814a2b80b772b07399cc58 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:13:56.551237  3346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e514dae1f1814a2b80b772b07399cc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.553097  3340 tablet_service.cc:1511] Processing CreateTablet for tablet 7c16129126724cb3a6e2ecfa3547b4e6 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:13:56.554811  3272 tablet_service.cc:1511] Processing CreateTablet for tablet f9ccb9fa24984b6c9a11b8429cd45338 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:13:56.555449  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c6fcbbfd7244c0a66649594e57fb03" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:56.555671  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c6fcbbfd7244c0a66649594e57fb03" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:56.556133  3272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9ccb9fa24984b6c9a11b8429cd45338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.556192  3282 raft_consensus.cc:2468] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:13:56.556548  3358 raft_consensus.cc:2393] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
I20260709 13:13:56.557507  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c6fcbbfd7244c0a66649594e57fb03" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:56.558144  3281 raft_consensus.cc:3060] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.558616  3505 raft_consensus.cc:2468] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:13:56.559166  3267 tablet_service.cc:1511] Processing CreateTablet for tablet 8ae6143fac5245deabc5979617a4f018 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:13:56.559959  3324 leader_election.cc:304] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:56.560400  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:56.560904  3562 raft_consensus.cc:2804] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:56.558141  3271 tablet_service.cc:1511] Processing CreateTablet for tablet 7c16129126724cb3a6e2ecfa3547b4e6 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:13:56.563885  3271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c16129126724cb3a6e2ecfa3547b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.565742  3281 raft_consensus.cc:2468] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:13:56.559944  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c16129126724cb3a6e2ecfa3547b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.570358  3418 tablet_service.cc:1511] Processing CreateTablet for tablet f9ccb9fa24984b6c9a11b8429cd45338 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:13:56.570966  3421 tablet_service.cc:1511] Processing CreateTablet for tablet ad53e24ea0f24ba6977b81fbb4608a83 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:13:56.571635  3418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9ccb9fa24984b6c9a11b8429cd45338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.572417  3421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad53e24ea0f24ba6977b81fbb4608a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.577059  3416 tablet_service.cc:1511] Processing CreateTablet for tablet 8ae6143fac5245deabc5979617a4f018 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:13:56.580255  3340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c16129126724cb3a6e2ecfa3547b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.582695  3344 tablet_service.cc:1511] Processing CreateTablet for tablet e1bb6275bf5a44debe9b7cdd0d757a3e (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:13:56.583930  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1bb6275bf5a44debe9b7cdd0d757a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.584352  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae6143fac5245deabc5979617a4f018. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:56.589356  3470 leader_election.cc:343] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.592231  3341 tablet_service.cc:1511] Processing CreateTablet for tablet 8ae6143fac5245deabc5979617a4f018 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:13:56.593573  3341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae6143fac5245deabc5979617a4f018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.593035  3347 tablet_service.cc:1511] Processing CreateTablet for tablet ad53e24ea0f24ba6977b81fbb4608a83 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:13:56.594534  3347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad53e24ea0f24ba6977b81fbb4608a83. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:56.595497  3323 leader_election.cc:343] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.595954  3323 leader_election.cc:304] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:56.601455  3553 raft_consensus.cc:359] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:56.602263  3553 raft_consensus.cc:385] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.602654  3553 raft_consensus.cc:740] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.608016  3416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae6143fac5245deabc5979617a4f018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.608582  3554 tablet_bootstrap.cc:654] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.625595  3496 tablet_service.cc:1511] Processing CreateTablet for tablet ecac352d2640492ba17f310cbb649d7c (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:13:56.632207  3552 consensus_queue.cc:260] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.635514  3552 ts_tablet_manager.cc:1434] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.103s	user 0.005s	sys 0.000s
I20260709 13:13:56.636845  3492 tablet_service.cc:1511] Processing CreateTablet for tablet ad53e24ea0f24ba6977b81fbb4608a83 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:13:56.672559  3552 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:56.672901  3492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad53e24ea0f24ba6977b81fbb4608a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.637696  3553 consensus_queue.cc:260] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:56.649501  3584 raft_consensus.cc:515] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:56.652488  3551 tablet_bootstrap.cc:654] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.670217  3420 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd7e8a9e3e248c4ba469d477a27f8cb (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:13:56.671133  3490 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd7e8a9e3e248c4ba469d477a27f8cb (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:13:56.683426  3420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd7e8a9e3e248c4ba469d477a27f8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.683514  3595 raft_consensus.cc:2749] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:56.680516  3562 raft_consensus.cc:697] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:13:56.690994  3496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecac352d2640492ba17f310cbb649d7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.690369  3562 consensus_queue.cc:237] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.697819  3584 leader_election.cc:290] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
W20260709 13:13:56.698236  3323 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 100456us
I20260709 13:13:56.699878  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cbb74eb76044f2a65c7a0b52c2adde" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:56.700528  3432 raft_consensus.cc:3060] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.701756  3273 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd7e8a9e3e248c4ba469d477a27f8cb (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:13:56.703039  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd7e8a9e3e248c4ba469d477a27f8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.706562  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd7e8a9e3e248c4ba469d477a27f8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.713145  3271 tablet_service.cc:1511] Processing CreateTablet for tablet ecac352d2640492ba17f310cbb649d7c (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:13:56.714840  3271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecac352d2640492ba17f310cbb649d7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.715147  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cbb74eb76044f2a65c7a0b52c2adde" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:56.715973  3358 raft_consensus.cc:3060] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.733892  3551 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:56.734469  3551 ts_tablet_manager.cc:1403] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.200s	user 0.019s	sys 0.003s
I20260709 13:13:56.743063  3552 tablet_bootstrap.cc:654] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.750614  3432 raft_consensus.cc:2468] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:56.751770  3470 leader_election.cc:304] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:13:56.746893  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c6fcbbfd7244c0a66649594e57fb03" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:56.752753  3281 raft_consensus.cc:2393] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
I20260709 13:13:56.752684  3584 raft_consensus.cc:2804] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:56.753304  3346 tablet_service.cc:1511] Processing CreateTablet for tablet ecac352d2640492ba17f310cbb649d7c (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:13:56.755039  3471 leader_election.cc:304] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:56.756363  3564 raft_consensus.cc:2749] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:56.753360  3584 raft_consensus.cc:697] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:13:56.758025  3584 consensus_queue.cc:237] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:56.761567  3344 tablet_service.cc:1511] Processing CreateTablet for tablet 85719491f2464bbf9c92b2f19238beeb (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:13:56.762861  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85719491f2464bbf9c92b2f19238beeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.746316  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cbb74eb76044f2a65c7a0b52c2adde" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:56.769969  3494 tablet_service.cc:1511] Processing CreateTablet for tablet 85719491f2464bbf9c92b2f19238beeb (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:13:56.771284  3494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85719491f2464bbf9c92b2f19238beeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.776191  3398 leader_election.cc:304] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:56.757472  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:56.781920  3506 raft_consensus.cc:2468] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:56.780500  3554 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:56.786154  3554 ts_tablet_manager.cc:1403] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.283s	user 0.012s	sys 0.041s
I20260709 13:13:56.786816  3418 tablet_service.cc:1511] Processing CreateTablet for tablet 85719491f2464bbf9c92b2f19238beeb (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:13:56.781270  3346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecac352d2640492ba17f310cbb649d7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.789055  3418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85719491f2464bbf9c92b2f19238beeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.793252  3267 tablet_service.cc:1511] Processing CreateTablet for tablet a1537c0aaa1d470e930915f178ec10d2 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:13:56.794617  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1537c0aaa1d470e930915f178ec10d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.795045  3558 raft_consensus.cc:493] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.795519  3558 raft_consensus.cc:515] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.796430  3358 raft_consensus.cc:2468] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:56.810242  3553 ts_tablet_manager.cc:1434] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.266s	user 0.006s	sys 0.022s
I20260709 13:13:56.813836  3554 raft_consensus.cc:359] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.814899  3554 raft_consensus.cc:385] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.815267  3554 raft_consensus.cc:740] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.821085  3347 tablet_service.cc:1511] Processing CreateTablet for tablet a1537c0aaa1d470e930915f178ec10d2 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:13:56.822461  3347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1537c0aaa1d470e930915f178ec10d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.829193  3558 leader_election.cc:290] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:56.811839  3553 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:56.837073  3559 raft_consensus.cc:2749] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:56.839968  3559 raft_consensus.cc:493] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.840555  3559 raft_consensus.cc:515] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:56.853675  3554 consensus_queue.cc:260] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.857290  3554 ts_tablet_manager.cc:1434] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.008s
I20260709 13:13:56.858333  3554 tablet_bootstrap.cc:492] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
W20260709 13:13:56.853300  3398 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 77351.7us
I20260709 13:13:56.860854  3398 leader_election.cc:304] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:13:56.862136  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:56.862846  3431 raft_consensus.cc:2468] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:56.864181  3559 leader_election.cc:290] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:56.864746  3559 raft_consensus.cc:2804] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:56.864967  3554 tablet_bootstrap.cc:654] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.865114  3559 raft_consensus.cc:493] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.865504  3559 raft_consensus.cc:3060] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.885861  3340 tablet_service.cc:1511] Processing CreateTablet for tablet 27a5322a6cb9422f85206f14c490fb6a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:13:56.887247  3340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a5322a6cb9422f85206f14c490fb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.888695  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:56.889556  3181 catalog_manager.cc:5697] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:56.890285  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:13:56.891639  3396 leader_election.cc:343] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:56.892185  3397 leader_election.cc:343] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:56.892652  3397 leader_election.cc:304] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:56.893394  3416 tablet_service.cc:1511] Processing CreateTablet for tablet a1537c0aaa1d470e930915f178ec10d2 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:13:56.894693  3416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1537c0aaa1d470e930915f178ec10d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.896916  3247 leader_election.cc:304] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:56.897013  3558 consensus_queue.cc:1048] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:56.899520  3558 raft_consensus.cc:2804] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:56.899878  3558 raft_consensus.cc:493] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.900218  3558 raft_consensus.cc:3060] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.903913  3606 raft_consensus.cc:2749] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:56.910398  3180 catalog_manager.cc:5697] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:56.917353  3551 raft_consensus.cc:359] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.918396  3551 raft_consensus.cc:385] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.918469  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:56.918994  3551 raft_consensus.cc:740] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.919262  3506 raft_consensus.cc:2468] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:56.920015  3551 consensus_queue.cc:260] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:56.920295  3267 tablet_service.cc:1511] Processing CreateTablet for tablet 27a5322a6cb9422f85206f14c490fb6a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:13:56.923120  3551 ts_tablet_manager.cc:1434] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.188s	user 0.004s	sys 0.004s
I20260709 13:13:56.940853  3492 tablet_service.cc:1511] Processing CreateTablet for tablet 27a5322a6cb9422f85206f14c490fb6a (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:13:57.006119  3249 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 65088.3us
I20260709 13:13:57.032951  3552 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:57.033494  3552 ts_tablet_manager.cc:1403] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.361s	user 0.024s	sys 0.008s
I20260709 13:13:57.037851  3418 tablet_service.cc:1511] Processing CreateTablet for tablet a9d414c916bc43f2a6579cf5fa1beaee (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:13:57.039386  3418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d414c916bc43f2a6579cf5fa1beaee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.030680  3551 tablet_bootstrap.cc:492] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:57.041285  3492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a5322a6cb9422f85206f14c490fb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.041302  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a5322a6cb9422f85206f14c490fb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.042649  3584 raft_consensus.cc:493] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.048941  3584 raft_consensus.cc:515] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.051606  3584 leader_election.cc:290] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:57.052779  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:13:57.054497  3471 leader_election.cc:343] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.054713  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
W20260709 13:13:57.056581  3470 leader_election.cc:343] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.057107  3470 leader_election.cc:304] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:57.057835  3558 raft_consensus.cc:515] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.062577  3584 raft_consensus.cc:2749] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:57.063160  3558 leader_election.cc:290] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:57.063428  3552 raft_consensus.cc:359] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.064122  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:57.064442  3552 raft_consensus.cc:385] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.064774  3506 raft_consensus.cc:3060] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.064924  3552 raft_consensus.cc:740] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.065974  3552 consensus_queue.cc:260] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.068590  3552 ts_tablet_manager.cc:1434] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.003s
I20260709 13:13:57.077895  3559 raft_consensus.cc:515] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.078236  3490 tablet_service.cc:1511] Processing CreateTablet for tablet a9d414c916bc43f2a6579cf5fa1beaee (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:13:57.079551  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d414c916bc43f2a6579cf5fa1beaee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.081072  3271 tablet_service.cc:1511] Processing CreateTablet for tablet a9d414c916bc43f2a6579cf5fa1beaee (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:13:57.081394  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:57.082433  3271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d414c916bc43f2a6579cf5fa1beaee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.082868  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:57.083729  3281 raft_consensus.cc:2393] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 06964934536148c2aa77a2f1c6382fea in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:13:57.088001  3397 leader_election.cc:304] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:57.089004  3606 raft_consensus.cc:2749] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:13:57.089813  3344 tablet_service.cc:1511] Processing CreateTablet for tablet 63dbef5a847a4451b6e4e46d15341334 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:13:57.098230  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63dbef5a847a4451b6e4e46d15341334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.091833  3273 tablet_service.cc:1511] Processing CreateTablet for tablet 22507d010a9a43f6834e5a2dfd53c148 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:13:57.092540  3559 leader_election.cc:290] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:57.093678  3552 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:57.094065  3506 raft_consensus.cc:2468] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:57.096853  3496 tablet_service.cc:1511] Processing CreateTablet for tablet 63dbef5a847a4451b6e4e46d15341334 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:13:57.109412  3496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63dbef5a847a4451b6e4e46d15341334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.112461  3249 leader_election.cc:304] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:57.113756  3607 raft_consensus.cc:2804] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.114599  3607 raft_consensus.cc:697] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:57.115522  3607 consensus_queue.cc:237] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.118462  3551 tablet_bootstrap.cc:654] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.091218  3420 tablet_service.cc:1511] Processing CreateTablet for tablet 63dbef5a847a4451b6e4e46d15341334 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:13:57.074942  3553 tablet_bootstrap.cc:654] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.120115  3420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63dbef5a847a4451b6e4e46d15341334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.101878  3554 tablet_bootstrap.cc:492] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:57.106712  3346 tablet_service.cc:1511] Processing CreateTablet for tablet 22507d010a9a43f6834e5a2dfd53c148 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:13:57.122298  3554 ts_tablet_manager.cc:1403] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.264s	user 0.011s	sys 0.023s
I20260709 13:13:57.108384  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22507d010a9a43f6834e5a2dfd53c148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.124683  3558 raft_consensus.cc:493] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.125216  3558 raft_consensus.cc:515] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.125716  3554 raft_consensus.cc:359] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.126526  3554 raft_consensus.cc:385] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.127339  3554 raft_consensus.cc:740] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.129431  3558 leader_election.cc:290] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:57.132516  3554 consensus_queue.cc:260] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.141296  3346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22507d010a9a43f6834e5a2dfd53c148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.146682  3552 tablet_bootstrap.cc:654] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.148232  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:57.149523  3249 leader_election.cc:343] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:57.152064  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 255171us
I20260709 13:13:57.156486  3562 consensus_queue.cc:1048] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:57.158322  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:57.159147  3432 raft_consensus.cc:2393] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 972a4469a4654416ab0a153d8c740282 in current term 1: Already voted for candidate 06964934536148c2aa77a2f1c6382fea in this term.
I20260709 13:13:57.171162  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:57.171846  3358 raft_consensus.cc:2468] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:57.172269  3554 ts_tablet_manager.cc:1434] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.002s
I20260709 13:13:57.173390  3554 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
W20260709 13:13:57.173444  3247 consensus_peers.cc:597] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:57.174360  3247 leader_election.cc:304] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:57.186688  3607 raft_consensus.cc:2804] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.187107  3607 raft_consensus.cc:493] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.187448  3607 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.189759  3273 tablet_service.cc:1511] Processing CreateTablet for tablet 6d122fabfab34d9ea6e8bf2100aaf1e3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:13:57.190701  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d122fabfab34d9ea6e8bf2100aaf1e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.181010  3418 tablet_service.cc:1511] Processing CreateTablet for tablet 22507d010a9a43f6834e5a2dfd53c148 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:13:57.207500  3562 consensus_queue.cc:1048] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:57.215095  3490 tablet_service.cc:1511] Processing CreateTablet for tablet f05be967181241fdb7f127f3505bbebc (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:13:57.216382  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05be967181241fdb7f127f3505bbebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.215018  3492 tablet_service.cc:1511] Processing CreateTablet for tablet 6d122fabfab34d9ea6e8bf2100aaf1e3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:13:57.197199  3418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22507d010a9a43f6834e5a2dfd53c148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.218026  3344 tablet_service.cc:1511] Processing CreateTablet for tablet 6d122fabfab34d9ea6e8bf2100aaf1e3 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:13:57.219563  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d122fabfab34d9ea6e8bf2100aaf1e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.226866  3551 tablet_bootstrap.cc:492] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:57.227419  3551 ts_tablet_manager.cc:1403] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.222s	user 0.014s	sys 0.030s
I20260709 13:13:57.230399  3551 raft_consensus.cc:359] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.231101  3551 raft_consensus.cc:385] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.231375  3551 raft_consensus.cc:740] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.227718  3267 tablet_service.cc:1511] Processing CreateTablet for tablet f05be967181241fdb7f127f3505bbebc (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:13:57.237757  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05be967181241fdb7f127f3505bbebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.240383  3492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d122fabfab34d9ea6e8bf2100aaf1e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.248728  3553 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:57.249257  3553 ts_tablet_manager.cc:1403] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.438s	user 0.017s	sys 0.009s
I20260709 13:13:57.251025  3344 tablet_service.cc:1511] Processing CreateTablet for tablet 03a915f11e804f04adb461457918ee75 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:13:57.252665  3344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a915f11e804f04adb461457918ee75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.215826  3607 raft_consensus.cc:515] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.247200  3178 catalog_manager.cc:5697] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:57.296875  3416 tablet_service.cc:1511] Processing CreateTablet for tablet f05be967181241fdb7f127f3505bbebc (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:13:57.298207  3416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05be967181241fdb7f127f3505bbebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.298363  3552 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:57.298914  3552 ts_tablet_manager.cc:1403] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.205s	user 0.015s	sys 0.007s
I20260709 13:13:57.300751  3273 tablet_service.cc:1511] Processing CreateTablet for tablet f2794d9308d0458aa11c33cb7dab1748 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:13:57.301872  3273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2794d9308d0458aa11c33cb7dab1748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.302347  3554 tablet_bootstrap.cc:654] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.303350  3553 raft_consensus.cc:359] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.305006  3271 tablet_service.cc:1511] Processing CreateTablet for tablet 40971e1ab1f04b21a166d7eac0ee7d95 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:13:57.306260  3271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40971e1ab1f04b21a166d7eac0ee7d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.298399  3420 tablet_service.cc:1511] Processing CreateTablet for tablet 03a915f11e804f04adb461457918ee75 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:13:57.307358  3420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a915f11e804f04adb461457918ee75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.308085  3553 raft_consensus.cc:385] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.308457  3553 raft_consensus.cc:740] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.309312  3553 consensus_queue.cc:260] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.313172  3607 leader_election.cc:290] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:57.315224  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:57.315827  3358 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.273387  3551 consensus_queue.cc:260] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.316893  3413 tablet_service.cc:1511] Processing CreateTablet for tablet f2794d9308d0458aa11c33cb7dab1748 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:13:57.318152  3413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2794d9308d0458aa11c33cb7dab1748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.320458  3347 tablet_service.cc:1511] Processing CreateTablet for tablet 40971e1ab1f04b21a166d7eac0ee7d95 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:13:57.335242  3347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40971e1ab1f04b21a166d7eac0ee7d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.336512  3559 raft_consensus.cc:493] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.337028  3559 raft_consensus.cc:515] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.339046  3559 leader_election.cc:290] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:57.320458  3340 tablet_service.cc:1511] Processing CreateTablet for tablet f2794d9308d0458aa11c33cb7dab1748 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:13:57.340610  3340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2794d9308d0458aa11c33cb7dab1748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.341935  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:57.344535  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:57.345842  3398 leader_election.cc:343] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.347782  3553 ts_tablet_manager.cc:1434] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.000s
I20260709 13:13:57.348881  3553 tablet_bootstrap.cc:492] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:57.336216  3496 tablet_service.cc:1511] Processing CreateTablet for tablet 40971e1ab1f04b21a166d7eac0ee7d95 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 13:13:57.359676  3249 leader_election.cc:343] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.360627  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:13:57.362025  3397 leader_election.cc:343] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.362462  3397 leader_election.cc:304] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:57.367004  3490 tablet_service.cc:1511] Processing CreateTablet for tablet 03a915f11e804f04adb461457918ee75 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:13:57.367238  3421 tablet_service.cc:1511] Processing CreateTablet for tablet e3a384f005d84258a11b79619f7d7ef4 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:13:57.368557  3421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a384f005d84258a11b79619f7d7ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.372568  3494 tablet_service.cc:1511] Processing CreateTablet for tablet e3a384f005d84258a11b79619f7d7ef4 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:13:57.376284  3564 consensus_queue.cc:1048] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:57.377643  3564 consensus_queue.cc:1048] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:57.379061  3496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40971e1ab1f04b21a166d7eac0ee7d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.379045  3358 raft_consensus.cc:2468] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:57.381489  3490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a915f11e804f04adb461457918ee75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.382478  3494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a384f005d84258a11b79619f7d7ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.391983  3267 tablet_service.cc:1511] Processing CreateTablet for tablet e3a384f005d84258a11b79619f7d7ef4 (DEFAULT_TABLE table=test-workload [id=261d2877058045279968cec5c7456fb8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:13:57.393254  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a384f005d84258a11b79619f7d7ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.395246  3247 leader_election.cc:304] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:57.391057  3559 raft_consensus.cc:2749] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:13:57.415701  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:57.417160  3553 tablet_bootstrap.cc:654] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.372442  3552 raft_consensus.cc:359] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.420444  3551 ts_tablet_manager.cc:1434] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.193s	user 0.004s	sys 0.008s
I20260709 13:13:57.439015  3551 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:57.424168  3607 raft_consensus.cc:2804] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.443674  3607 raft_consensus.cc:697] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:57.437853  3552 raft_consensus.cc:385] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.454010  3552 raft_consensus.cc:740] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.457144  3552 consensus_queue.cc:260] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.459533  3552 ts_tablet_manager.cc:1434] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.160s	user 0.008s	sys 0.002s
I20260709 13:13:57.454560  3607 consensus_queue.cc:237] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.471941  3551 tablet_bootstrap.cc:654] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.476383  3552 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:57.493747  3564 raft_consensus.cc:493] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.494297  3564 raft_consensus.cc:515] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.503021  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:57.504426  3564 leader_election.cc:290] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:57.507128  3553 tablet_bootstrap.cc:492] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:57.513121  3623 raft_consensus.cc:493] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.513827  3623 raft_consensus.cc:515] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.521903  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
W20260709 13:13:57.521459  3470 leader_election.cc:343] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.523288  3623 leader_election.cc:290] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:57.524564  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3061a90b8dbc42488cec94032788e597" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:57.526330  3553 ts_tablet_manager.cc:1403] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.178s	user 0.015s	sys 0.004s
I20260709 13:13:57.538852  3178 catalog_manager.cc:5697] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:57.550588  3552 tablet_bootstrap.cc:654] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.550148  3553 raft_consensus.cc:359] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.551162  3553 raft_consensus.cc:385] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.551566  3553 raft_consensus.cc:740] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
W20260709 13:13:57.555159  3323 leader_election.cc:343] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.558990  3559 raft_consensus.cc:493] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.559474  3559 raft_consensus.cc:515] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.561499  3559 leader_election.cc:290] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:57.557822  3553 consensus_queue.cc:260] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.562582  3620 consensus_queue.cc:1048] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:57.533794  3470 leader_election.cc:343] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.565379  3470 leader_election.cc:304] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:57.566620  3629 raft_consensus.cc:2749] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:57.567358  3553 ts_tablet_manager.cc:1434] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:13:57.570596  3551 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:57.571127  3551 ts_tablet_manager.cc:1403] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.132s	user 0.028s	sys 0.022s
I20260709 13:13:57.571298  3554 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:57.571815  3554 ts_tablet_manager.cc:1403] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.399s	user 0.016s	sys 0.017s
I20260709 13:13:57.574162  3551 raft_consensus.cc:359] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.579403  3551 raft_consensus.cc:385] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.579763  3551 raft_consensus.cc:740] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.580686  3551 consensus_queue.cc:260] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.583236  3551 ts_tablet_manager.cc:1434] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:13:57.584316  3551 tablet_bootstrap.cc:492] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:57.568514  3553 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:57.576011  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514a52a2733c4c2f996475e12c2dbaa8" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:57.577088  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514a52a2733c4c2f996475e12c2dbaa8" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:57.587607  3357 raft_consensus.cc:2468] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:57.588753  3396 leader_election.cc:304] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:57.589618  3559 raft_consensus.cc:2804] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.589978  3559 raft_consensus.cc:493] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.590291  3559 raft_consensus.cc:3060] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:57.591598  3398 leader_election.cc:343] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.583345  3554 raft_consensus.cc:359] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.592329  3554 raft_consensus.cc:385] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.592636  3554 raft_consensus.cc:740] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.592865  3551 tablet_bootstrap.cc:654] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.588766  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3061a90b8dbc42488cec94032788e597" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
W20260709 13:13:57.598014  3322 leader_election.cc:343] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.598470  3322 leader_election.cc:304] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:57.602386  3553 tablet_bootstrap.cc:654] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.604228  3623 raft_consensus.cc:2749] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:57.599349  3554 consensus_queue.cc:260] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.603816  3559 raft_consensus.cc:515] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.612725  3620 consensus_queue.cc:1048] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:57.620487  3554 ts_tablet_manager.cc:1434] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:13:57.621871  3554 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:57.638986  3629 raft_consensus.cc:493] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.639740  3629 raft_consensus.cc:515] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.643097  3629 leader_election.cc:290] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:57.644205  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:57.644887  3281 raft_consensus.cc:2468] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:57.646391  3552 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:57.646277  3471 leader_election.cc:304] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:57.646997  3552 ts_tablet_manager.cc:1403] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.181s	user 0.014s	sys 0.004s
I20260709 13:13:57.647559  3629 raft_consensus.cc:2804] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.648873  3629 raft_consensus.cc:493] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.649282  3629 raft_consensus.cc:3060] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.650238  3554 tablet_bootstrap.cc:654] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.648303  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:57.650079  3552 raft_consensus.cc:359] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.651012  3552 raft_consensus.cc:385] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.651350  3552 raft_consensus.cc:740] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
W20260709 13:13:57.651818  3470 leader_election.cc:343] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.652190  3552 consensus_queue.cc:260] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.652992  3559 leader_election.cc:290] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:57.654728  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514a52a2733c4c2f996475e12c2dbaa8" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:57.655344  3357 raft_consensus.cc:3060] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.655372  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514a52a2733c4c2f996475e12c2dbaa8" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
W20260709 13:13:57.657163  3398 leader_election.cc:343] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.662237  3552 ts_tablet_manager.cc:1434] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.004s
I20260709 13:13:57.663600  3552 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:57.665928  3551 tablet_bootstrap.cc:492] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:57.666482  3551 ts_tablet_manager.cc:1403] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.026s
I20260709 13:13:57.666582  3357 raft_consensus.cc:2468] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:57.668740  3396 leader_election.cc:304] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:57.669617  3553 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:57.669340  3551 raft_consensus.cc:359] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.670063  3551 raft_consensus.cc:385] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.670075  3559 raft_consensus.cc:2804] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.670890  3559 raft_consensus.cc:697] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:13:57.671463  3629 raft_consensus.cc:515] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.670120  3553 ts_tablet_manager.cc:1403] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.102s	user 0.024s	sys 0.009s
I20260709 13:13:57.675824  3553 raft_consensus.cc:359] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.671782  3559 consensus_queue.cc:237] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.678692  3552 tablet_bootstrap.cc:654] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.679642  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:57.680559  3553 raft_consensus.cc:385] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.680981  3553 raft_consensus.cc:740] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
W20260709 13:13:57.681118  3470 leader_election.cc:343] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.682017  3553 consensus_queue.cc:260] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.683643  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:57.684260  3281 raft_consensus.cc:3060] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.685520  3554 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:57.686045  3554 ts_tablet_manager.cc:1403] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.000s
I20260709 13:13:57.689163  3554 raft_consensus.cc:359] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.689965  3554 raft_consensus.cc:385] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.692279  3281 raft_consensus.cc:2468] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:57.670536  3551 raft_consensus.cc:740] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.692852  3629 leader_election.cc:290] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:57.694617  3471 leader_election.cc:304] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 06964934536148c2aa77a2f1c6382fea
I20260709 13:13:57.695561  3564 raft_consensus.cc:2804] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.696120  3564 raft_consensus.cc:697] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:13:57.685518  3553 ts_tablet_manager.cc:1434] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 13:13:57.699189  3551 consensus_queue.cc:260] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.699955  3564 consensus_queue.cc:237] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.701892  3551 ts_tablet_manager.cc:1434] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.002s
I20260709 13:13:57.702735  3553 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:57.702868  3551 tablet_bootstrap.cc:492] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:57.690275  3554 raft_consensus.cc:740] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.708736  3554 consensus_queue.cc:260] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.711051  3554 ts_tablet_manager.cc:1434] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 13:13:57.712114  3554 tablet_bootstrap.cc:492] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:57.712970  3551 tablet_bootstrap.cc:654] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.715476  3179 catalog_manager.cc:5697] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:57.720822  3552 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:57.721361  3552 ts_tablet_manager.cc:1403] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.001s
I20260709 13:13:57.724174  3554 tablet_bootstrap.cc:654] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.724301  3552 raft_consensus.cc:359] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.724997  3552 raft_consensus.cc:385] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.725304  3552 raft_consensus.cc:740] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.726174  3552 consensus_queue.cc:260] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.727451  3551 tablet_bootstrap.cc:492] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:57.728137  3551 ts_tablet_manager.cc:1403] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.000s
I20260709 13:13:57.728168  3552 ts_tablet_manager.cc:1434] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:13:57.728307  3181 catalog_manager.cc:5697] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:57.729794  3552 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:57.730841  3551 raft_consensus.cc:359] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.731503  3551 raft_consensus.cc:385] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.731750  3551 raft_consensus.cc:740] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.732389  3551 consensus_queue.cc:260] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.734822  3551 ts_tablet_manager.cc:1434] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:13:57.735677  3551 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:57.735909  3552 tablet_bootstrap.cc:654] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.747151  3553 tablet_bootstrap.cc:654] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.748126  3551 tablet_bootstrap.cc:654] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.760231  3552 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:57.760741  3552 ts_tablet_manager.cc:1403] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.012s
I20260709 13:13:57.763571  3552 raft_consensus.cc:359] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.764295  3552 raft_consensus.cc:385] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.764583  3552 raft_consensus.cc:740] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.765334  3552 consensus_queue.cc:260] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.765856  3554 tablet_bootstrap.cc:492] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:57.766505  3554 ts_tablet_manager.cc:1403] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.005s
I20260709 13:13:57.769292  3554 raft_consensus.cc:359] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.770015  3554 raft_consensus.cc:385] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.770350  3554 raft_consensus.cc:740] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.771159  3554 consensus_queue.cc:260] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.771601  3552 ts_tablet_manager.cc:1434] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:13:57.772687  3552 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:57.778631  3552 tablet_bootstrap.cc:654] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.782285  3553 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:57.782829  3553 ts_tablet_manager.cc:1403] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.011s
I20260709 13:13:57.785295  3554 ts_tablet_manager.cc:1434] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:13:57.785830  3553 raft_consensus.cc:359] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.786545  3553 raft_consensus.cc:385] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.786907  3553 raft_consensus.cc:740] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.788537  3553 consensus_queue.cc:260] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.789851  3551 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:57.790369  3551 ts_tablet_manager.cc:1403] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.055s	user 0.007s	sys 0.004s
I20260709 13:13:57.791141  3553 ts_tablet_manager.cc:1434] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:13:57.792200  3553 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:57.793423  3551 raft_consensus.cc:359] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.794149  3551 raft_consensus.cc:385] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.794432  3551 raft_consensus.cc:740] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.794981  3554 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:57.795234  3551 consensus_queue.cc:260] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.801563  3554 tablet_bootstrap.cc:654] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.801945  3551 ts_tablet_manager.cc:1434] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:13:57.803067  3551 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:57.814363  3554 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:57.814980  3554 ts_tablet_manager.cc:1403] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:13:57.818995  3553 tablet_bootstrap.cc:654] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.820443  3552 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:57.820986  3552 ts_tablet_manager.cc:1403] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.000s
I20260709 13:13:57.824667  3551 tablet_bootstrap.cc:654] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.817986  3554 raft_consensus.cc:359] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.825764  3554 raft_consensus.cc:385] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.826202  3554 raft_consensus.cc:740] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.824579  3552 raft_consensus.cc:359] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.827790  3552 raft_consensus.cc:385] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.829138  3552 raft_consensus.cc:740] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.830351  3553 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:57.828387  3554 consensus_queue.cc:260] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.830839  3553 ts_tablet_manager.cc:1403] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.039s	user 0.008s	sys 0.008s
I20260709 13:13:57.833650  3553 raft_consensus.cc:359] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.834288  3553 raft_consensus.cc:385] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.834538  3553 raft_consensus.cc:740] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.829988  3552 consensus_queue.cc:260] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.836822  3552 ts_tablet_manager.cc:1434] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:13:57.837409  3553 consensus_queue.cc:260] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.839268  3552 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:57.842489  3553 ts_tablet_manager.cc:1434] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:13:57.843567  3553 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:57.849503  3553 tablet_bootstrap.cc:654] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.849524  3552 tablet_bootstrap.cc:654] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.851539  3551 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:57.852016  3551 ts_tablet_manager.cc:1403] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.003s
I20260709 13:13:57.855677  3551 raft_consensus.cc:359] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.856614  3551 raft_consensus.cc:385] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.856973  3551 raft_consensus.cc:740] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.857869  3551 consensus_queue.cc:260] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.860165  3551 ts_tablet_manager.cc:1434] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:13:57.861578  3559 raft_consensus.cc:493] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.861577  3553 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:57.862604  3553 ts_tablet_manager.cc:1403] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:13:57.862172  3559 raft_consensus.cc:515] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.864744  3551 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:57.865723  3553 raft_consensus.cc:359] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.866215  3554 ts_tablet_manager.cc:1434] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.001s
I20260709 13:13:57.866413  3553 raft_consensus.cc:385] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.866746  3553 raft_consensus.cc:740] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.867574  3553 consensus_queue.cc:260] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.869570  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:57.869830  3553 ts_tablet_manager.cc:1434] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:13:57.870199  3505 raft_consensus.cc:2468] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:57.870786  3553 tablet_bootstrap.cc:492] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:57.871024  3551 tablet_bootstrap.cc:654] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.871870  3398 leader_election.cc:304] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:13:57.871912  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:57.872792  3281 raft_consensus.cc:2468] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:57.872928  3606 raft_consensus.cc:2804] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.873379  3606 raft_consensus.cc:493] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.873725  3606 raft_consensus.cc:3060] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.876477  3553 tablet_bootstrap.cc:654] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.878388  3505 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.883741  3623 raft_consensus.cc:493] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.884279  3623 raft_consensus.cc:515] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.886299  3623 leader_election.cc:290] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:57.864543  3559 leader_election.cc:290] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:57.887768  3559 raft_consensus.cc:493] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.888836  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:57.888439  3559 raft_consensus.cc:515] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.891472  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:57.896384  3322 leader_election.cc:304] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:57.897894  3623 raft_consensus.cc:2749] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:57.898638  3564 raft_consensus.cc:493] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.899271  3564 raft_consensus.cc:515] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.901433  3564 leader_election.cc:290] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:57.902531  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d04edf84bba4d759de41bab8b424463" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:57.903168  3357 raft_consensus.cc:2468] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:57.903560  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:57.903853  3559 leader_election.cc:290] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:57.904466  3559 raft_consensus.cc:493] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.904877  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d04edf84bba4d759de41bab8b424463" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:57.904945  3470 leader_election.cc:304] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:57.905656  3281 raft_consensus.cc:2468] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:57.905965  3564 raft_consensus.cc:2804] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.906356  3564 raft_consensus.cc:493] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.906674  3564 raft_consensus.cc:3060] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.908254  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:57.909015  3281 raft_consensus.cc:2393] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06964934536148c2aa77a2f1c6382fea in current term 1: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:13:57.908829  3606 raft_consensus.cc:515] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:57.910607  3398 leader_election.cc:304] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:57.911120  3606 leader_election.cc:290] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:57.911743  3606 raft_consensus.cc:3060] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.913729  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:57.914354  3506 raft_consensus.cc:3060] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.915287  3552 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:57.916363  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:57.916946  3281 raft_consensus.cc:3060] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.917765  3554 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:57.915934  3552 ts_tablet_manager.cc:1403] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.007s
I20260709 13:13:57.921957  3614 consensus_queue.cc:1048] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:57.904892  3559 raft_consensus.cc:515] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:57.934228  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb78494b2e0410ba8b06a30fff7158a" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:57.935492  3554 tablet_bootstrap.cc:654] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:57.935989  3396 leader_election.cc:343] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.936548  3564 raft_consensus.cc:515] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.940176  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d04edf84bba4d759de41bab8b424463" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:57.940733  3283 raft_consensus.cc:3060] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.945371  3558 consensus_queue.cc:1048] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:57.948971  3564 leader_election.cc:290] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:57.950428  3283 raft_consensus.cc:2468] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:57.950903  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d04edf84bba4d759de41bab8b424463" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:57.951500  3358 raft_consensus.cc:3060] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.951620  3471 leader_election.cc:304] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:57.952623  3629 raft_consensus.cc:2804] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.953080  3629 raft_consensus.cc:697] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:13:57.953914  3629 consensus_queue.cc:237] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:57.955052  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb78494b2e0410ba8b06a30fff7158a" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:57.956305  3398 leader_election.cc:343] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.956760  3398 leader_election.cc:304] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:57.957926  3653 raft_consensus.cc:2749] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:57.957579  3552 raft_consensus.cc:359] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.958488  3552 raft_consensus.cc:385] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.958817  3552 raft_consensus.cc:740] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.959318  3506 raft_consensus.cc:2468] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:57.960533  3398 leader_election.cc:304] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:13:57.962368  3653 raft_consensus.cc:2804] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.962845  3552 consensus_queue.cc:260] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.965559  3551 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:57.966037  3551 ts_tablet_manager.cc:1403] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.101s	user 0.017s	sys 0.025s
I20260709 13:13:57.969723  3552 ts_tablet_manager.cc:1434] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.003s
I20260709 13:13:57.970952  3552 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:57.975616  3358 raft_consensus.cc:2468] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:57.962752  3559 leader_election.cc:290] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:57.977464  3551 raft_consensus.cc:359] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.978269  3551 raft_consensus.cc:385] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.978549  3551 raft_consensus.cc:740] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.979229  3551 consensus_queue.cc:260] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:57.981164  3551 ts_tablet_manager.cc:1434] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:13:57.985386  3281 raft_consensus.cc:2468] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:57.988402  3358 raft_consensus.cc:3060] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.982102  3551 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:57.999478  3553 tablet_bootstrap.cc:492] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:58.000142  3553 ts_tablet_manager.cc:1403] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.130s	user 0.022s	sys 0.003s
I20260709 13:13:58.002813  3551 tablet_bootstrap.cc:654] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.003715  3553 raft_consensus.cc:359] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.004580  3553 raft_consensus.cc:385] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.004906  3553 raft_consensus.cc:740] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.007040  3606 raft_consensus.cc:2749] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.006901  3553 consensus_queue.cc:260] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.008987  3552 tablet_bootstrap.cc:654] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.012635  3553 ts_tablet_manager.cc:1434] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.004s
I20260709 13:13:58.013864  3553 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:58.011276  3653 raft_consensus.cc:697] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:13:58.016812  3653 consensus_queue.cc:237] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.022372  3554 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:58.024566  3554 ts_tablet_manager.cc:1403] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.107s	user 0.012s	sys 0.011s
I20260709 13:13:58.022563  3553 tablet_bootstrap.cc:654] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.022637  3179 catalog_manager.cc:5697] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.029819  3554 raft_consensus.cc:359] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.030719  3554 raft_consensus.cc:385] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.031042  3554 raft_consensus.cc:740] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.027781  3660 raft_consensus.cc:493] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.031782  3554 consensus_queue.cc:260] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.031868  3660 raft_consensus.cc:515] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.036266  3551 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.038179  3660 leader_election.cc:290] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:58.039108  3653 raft_consensus.cc:493] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.039647  3653 raft_consensus.cc:515] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.040747  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3061a90b8dbc42488cec94032788e597" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:58.041427  3281 raft_consensus.cc:2468] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:58.041659  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3061a90b8dbc42488cec94032788e597" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.042276  3357 raft_consensus.cc:2468] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:58.043558  3396 leader_election.cc:304] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:58.044648  3660 raft_consensus.cc:2804] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.045048  3660 raft_consensus.cc:493] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.045147  3623 raft_consensus.cc:493] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.045428  3660 raft_consensus.cc:3060] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.045806  3623 raft_consensus.cc:515] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.048311  3623 leader_election.cc:290] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:58.044122  3551 ts_tablet_manager.cc:1403] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.062s	user 0.010s	sys 0.002s
I20260709 13:13:58.053910  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "018917aa98ef46b4990e809f93148c34" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:58.054876  3281 raft_consensus.cc:2468] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:13:58.056020  3551 raft_consensus.cc:359] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.056732  3551 raft_consensus.cc:385] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.057039  3551 raft_consensus.cc:740] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.058264  3653 leader_election.cc:290] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:58.057911  3178 catalog_manager.cc:5697] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.059046  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.059739  3357 raft_consensus.cc:2468] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:58.064771  3551 consensus_queue.cc:260] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.066583  3323 leader_election.cc:304] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:58.066689  3396 leader_election.cc:304] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:58.067415  3554 ts_tablet_manager.cc:1434] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.002s
I20260709 13:13:58.068814  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:58.070791  3506 raft_consensus.cc:2468] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:58.071671  3606 raft_consensus.cc:2804] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.056746  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "018917aa98ef46b4990e809f93148c34" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:58.072096  3606 raft_consensus.cc:493] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.072459  3606 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.072434  3431 raft_consensus.cc:2468] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:13:58.079783  3558 raft_consensus.cc:493] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.080374  3558 raft_consensus.cc:515] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.083555  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:58.083755  3558 leader_election.cc:290] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
W20260709 13:13:58.085119  3249 leader_election.cc:343] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.087165  3554 tablet_bootstrap.cc:492] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:58.090787  3551 ts_tablet_manager.cc:1434] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 13:13:58.091809  3551 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.092453  3660 raft_consensus.cc:515] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.088011  3559 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.087834  3623 raft_consensus.cc:2804] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.089422  3558 raft_consensus.cc:493] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.096235  3623 raft_consensus.cc:493] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.096663  3623 raft_consensus.cc:3060] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.098838  3606 raft_consensus.cc:515] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.100149  3559 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.102447  3559 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:58.102876  3552 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:58.103408  3552 ts_tablet_manager.cc:1403] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.133s	user 0.027s	sys 0.024s
I20260709 13:13:58.103644  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:13:58.105588  3397 leader_election.cc:343] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.107890  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:58.107890  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.108718  3431 raft_consensus.cc:2468] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:58.108949  3551 tablet_bootstrap.cc:654] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:58.109555  3396 leader_election.cc:343] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.110036  3396 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:58.096637  3558 raft_consensus.cc:515] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.110036  3247 leader_election.cc:304] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:58.112234  3620 raft_consensus.cc:2804] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.112614  3620 raft_consensus.cc:493] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.112648  3558 leader_election.cc:290] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:58.113098  3620 raft_consensus.cc:3060] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.118999  3552 raft_consensus.cc:359] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.119726  3552 raft_consensus.cc:385] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.119984  3552 raft_consensus.cc:740] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.120448  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.120658  3552 consensus_queue.cc:260] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.121069  3358 raft_consensus.cc:2468] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:58.126984  3660 leader_election.cc:290] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:58.129951  3247 leader_election.cc:304] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:58.130555  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3061a90b8dbc42488cec94032788e597" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:58.131206  3281 raft_consensus.cc:3060] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.133428  3553 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:58.133958  3553 ts_tablet_manager.cc:1403] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.121s	user 0.009s	sys 0.014s
I20260709 13:13:58.128487  3559 raft_consensus.cc:2749] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.137068  3558 raft_consensus.cc:2804] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.138684  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:58.139401  3431 raft_consensus.cc:2468] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:58.139909  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:58.140506  3357 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.140558  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3061a90b8dbc42488cec94032788e597" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:58.141530  3358 raft_consensus.cc:3060] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.144554  3553 raft_consensus.cc:359] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.145288  3553 raft_consensus.cc:385] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.145606  3553 raft_consensus.cc:740] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.147580  3553 consensus_queue.cc:260] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.151948  3552 ts_tablet_manager.cc:1434] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 13:13:58.138774  3558 raft_consensus.cc:493] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.152791  3558 raft_consensus.cc:3060] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.153369  3552 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:58.150291  3554 tablet_bootstrap.cc:654] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.160264  3620 raft_consensus.cc:515] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.166428  3551 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.167086  3551 ts_tablet_manager.cc:1403] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.075s	user 0.017s	sys 0.000s
I20260709 13:13:58.173184  3506 raft_consensus.cc:3060] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.163947  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:58.174348  3431 raft_consensus.cc:3060] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.180967  3564 consensus_queue.cc:1048] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.184442  3623 raft_consensus.cc:515] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.186625  3623 leader_election.cc:290] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:58.188150  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "018917aa98ef46b4990e809f93148c34" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:58.188787  3432 raft_consensus.cc:3060] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.192250  3558 raft_consensus.cc:515] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.163034  3357 raft_consensus.cc:2468] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:58.192446  3551 raft_consensus.cc:359] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.194643  3553 ts_tablet_manager.cc:1434] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.060s	user 0.012s	sys 0.011s
I20260709 13:13:58.195169  3551 raft_consensus.cc:385] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.195611  3551 raft_consensus.cc:740] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.196139  3553 tablet_bootstrap.cc:492] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:58.198686  3551 consensus_queue.cc:260] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.202041  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
W20260709 13:13:58.203650  3249 leader_election.cc:343] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.193457  3660 consensus_queue.cc:1048] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:13:58.204613  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:58.205181  3505 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.194591  3606 leader_election.cc:290] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:58.210223  3551 ts_tablet_manager.cc:1434] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.002s
I20260709 13:13:58.211225  3551 tablet_bootstrap.cc:492] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.212875  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "018917aa98ef46b4990e809f93148c34" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:58.213491  3282 raft_consensus.cc:3060] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.216889  3396 leader_election.cc:304] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:58.208261  3432 raft_consensus.cc:2468] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:13:58.218289  3322 leader_election.cc:304] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:58.219672  3358 raft_consensus.cc:2468] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:58.221029  3431 raft_consensus.cc:2468] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:58.222313  3247 leader_election.cc:304] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:58.223204  3607 raft_consensus.cc:2804] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.223670  3607 raft_consensus.cc:697] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:58.224628  3607 consensus_queue.cc:237] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.225193  3396 leader_election.cc:304] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:58.229395  3505 raft_consensus.cc:2468] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:58.229795  3660 raft_consensus.cc:2804] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.230492  3559 raft_consensus.cc:2804] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.230976  3559 raft_consensus.cc:697] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:13:58.231806  3559 consensus_queue.cc:237] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.235064  3660 raft_consensus.cc:697] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:13:58.236182  3660 consensus_queue.cc:237] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.236515  3553 tablet_bootstrap.cc:654] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.207813  3620 leader_election.cc:290] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:58.207645  3281 raft_consensus.cc:2468] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:58.242491  3584 raft_consensus.cc:493] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.241326  3668 raft_consensus.cc:2804] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.256170  3282 raft_consensus.cc:2468] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:13:58.251197  3584 raft_consensus.cc:515] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.259707  3629 consensus_queue.cc:1048] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:58.258061  3668 raft_consensus.cc:697] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:13:58.262362  3668 consensus_queue.cc:237] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.266733  3551 tablet_bootstrap.cc:654] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.269045  3554 tablet_bootstrap.cc:492] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:58.269536  3554 ts_tablet_manager.cc:1403] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.183s	user 0.012s	sys 0.016s
I20260709 13:13:58.275142  3558 leader_election.cc:290] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:58.277194  3554 raft_consensus.cc:359] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.277740  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:58.277980  3554 raft_consensus.cc:385] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.278317  3554 raft_consensus.cc:740] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.278388  3432 raft_consensus.cc:3060] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.279203  3554 consensus_queue.cc:260] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.264581  3559 consensus_queue.cc:1048] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:13:58.280320  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:58.280854  3358 raft_consensus.cc:3060] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.278525  3552 tablet_bootstrap.cc:654] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.289114  3554 ts_tablet_manager.cc:1434] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.003s
I20260709 13:13:58.290354  3554 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:58.308154  3584 leader_election.cc:290] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:58.310113  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514a52a2733c4c2f996475e12c2dbaa8" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:58.310307  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514a52a2733c4c2f996475e12c2dbaa8" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.313133  3178 catalog_manager.cc:5697] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.314065  3470 leader_election.cc:304] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:58.315213  3432 raft_consensus.cc:2468] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:58.315652  3584 raft_consensus.cc:2749] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:58.316977  3247 leader_election.cc:304] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:58.317806  3607 raft_consensus.cc:2804] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.318967  3607 raft_consensus.cc:697] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:58.327260  3607 consensus_queue.cc:237] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.328555  3553 tablet_bootstrap.cc:492] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:58.329017  3553 ts_tablet_manager.cc:1403] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.133s	user 0.014s	sys 0.013s
I20260709 13:13:58.314164  3358 raft_consensus.cc:2468] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:58.337091  3553 raft_consensus.cc:359] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.337913  3553 raft_consensus.cc:385] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.338326  3551 tablet_bootstrap.cc:492] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.338323  3553 raft_consensus.cc:740] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.338939  3551 ts_tablet_manager.cc:1403] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.128s	user 0.010s	sys 0.010s
I20260709 13:13:58.339494  3553 consensus_queue.cc:260] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.341933  3553 ts_tablet_manager.cc:1434] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:13:58.341805  3551 raft_consensus.cc:359] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.345086  3551 raft_consensus.cc:385] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.345103  3553 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:58.345466  3551 raft_consensus.cc:740] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.347635  3551 consensus_queue.cc:260] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.350492  3551 ts_tablet_manager.cc:1434] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:13:58.351548  3551 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.358119  3179 catalog_manager.cc:5697] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.384759  3606 raft_consensus.cc:493] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.385551  3606 raft_consensus.cc:515] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.389175  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:13:58.390581  3396 leader_election.cc:343] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.393191  3668 raft_consensus.cc:493] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.393718  3668 raft_consensus.cc:515] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.395767  3668 leader_election.cc:290] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:58.397982  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e6844fd8644c6498d5b9ce0e7cbd6b" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:13:58.399451  3323 leader_election.cc:343] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.401959  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e6844fd8644c6498d5b9ce0e7cbd6b" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:58.403442  3324 leader_election.cc:343] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.403898  3324 leader_election.cc:304] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:58.405085  3668 raft_consensus.cc:2749] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.406040  3606 leader_election.cc:290] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:58.408200  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:58.409884  3398 leader_election.cc:343] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.410451  3398 leader_election.cc:304] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:58.412086  3660 raft_consensus.cc:2749] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.424525  3552 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:58.425184  3552 ts_tablet_manager.cc:1403] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.272s	user 0.013s	sys 0.023s
I20260709 13:13:58.429270  3552 raft_consensus.cc:359] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.430069  3552 raft_consensus.cc:385] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.430397  3552 raft_consensus.cc:740] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.431113  3552 consensus_queue.cc:260] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.433432  3552 ts_tablet_manager.cc:1434] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:13:58.434464  3552 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:58.434207  3178 catalog_manager.cc:5697] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.437757  3551 tablet_bootstrap.cc:654] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.440001  3553 tablet_bootstrap.cc:654] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:58.440225  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:58.440482  3564 consensus_queue.cc:1048] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.405084  3179 catalog_manager.cc:5697] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.450670  3554 tablet_bootstrap.cc:654] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.456022  3551 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.456545  3551 ts_tablet_manager.cc:1403] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.003s
I20260709 13:13:58.453887  3564 raft_consensus.cc:493] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.457985  3564 raft_consensus.cc:515] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.459216  3551 raft_consensus.cc:359] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.459872  3551 raft_consensus.cc:385] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.460009  3564 leader_election.cc:290] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:58.460402  3551 raft_consensus.cc:740] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.462121  3553 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:58.462580  3553 ts_tablet_manager.cc:1403] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.118s	user 0.011s	sys 0.006s
I20260709 13:13:58.464797  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb78494b2e0410ba8b06a30fff7158a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:58.465480  3431 raft_consensus.cc:2468] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:58.466434  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb78494b2e0410ba8b06a30fff7158a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.467061  3357 raft_consensus.cc:2468] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:58.467417  3552 tablet_bootstrap.cc:654] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.468348  3470 leader_election.cc:304] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:58.469228  3564 raft_consensus.cc:2804] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.469681  3564 raft_consensus.cc:493] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.470134  3564 raft_consensus.cc:3060] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.484995  3551 consensus_queue.cc:260] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.494057  3551 ts_tablet_manager.cc:1434] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:13:58.494874  3551 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.498242  3629 consensus_queue.cc:1048] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:13:58.502643  3553 raft_consensus.cc:359] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.503324  3553 raft_consensus.cc:385] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.503618  3553 raft_consensus.cc:740] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.504463  3553 consensus_queue.cc:260] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.507052  3553 ts_tablet_manager.cc:1434] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.044s	user 0.003s	sys 0.004s
I20260709 13:13:58.508199  3553 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:58.519177  3554 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:58.519765  3554 ts_tablet_manager.cc:1403] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.230s	user 0.007s	sys 0.010s
I20260709 13:13:58.528360  3629 raft_consensus.cc:493] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.529006  3629 raft_consensus.cc:515] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.532615  3629 leader_election.cc:290] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:58.532824  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a468aba2a1664f7b8a5e1755775a2e72" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.533522  3358 raft_consensus.cc:2468] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:58.534688  3470 leader_election.cc:304] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:58.534960  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a468aba2a1664f7b8a5e1755775a2e72" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:58.535576  3282 raft_consensus.cc:2468] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:58.538251  3629 raft_consensus.cc:2804] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.538750  3629 raft_consensus.cc:493] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.539135  3629 raft_consensus.cc:3060] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.545686  3554 raft_consensus.cc:359] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.546522  3554 raft_consensus.cc:385] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.546872  3554 raft_consensus.cc:740] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.547720  3554 consensus_queue.cc:260] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.543891  3551 tablet_bootstrap.cc:654] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.550160  3181 catalog_manager.cc:5697] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.561053  3554 ts_tablet_manager.cc:1434] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 13:13:58.562301  3554 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:58.515852  3564 raft_consensus.cc:515] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.571956  3559 raft_consensus.cc:493] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.572491  3559 raft_consensus.cc:515] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.574544  3559 leader_election.cc:290] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:58.565707  3553 tablet_bootstrap.cc:654] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.576843  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:58.579272  3552 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:58.580860  3552 ts_tablet_manager.cc:1403] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.147s	user 0.012s	sys 0.022s
W20260709 13:13:58.580943  3398 leader_election.cc:343] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.582129  3564 leader_election.cc:290] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:58.582772  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb78494b2e0410ba8b06a30fff7158a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:58.583359  3358 raft_consensus.cc:3060] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.583609  3552 raft_consensus.cc:359] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.571280  3629 raft_consensus.cc:515] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.579874  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:58.584316  3552 raft_consensus.cc:385] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.584642  3552 raft_consensus.cc:740] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
W20260709 13:13:58.585382  3397 leader_election.cc:343] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.585310  3552 consensus_queue.cc:260] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.585877  3397 leader_election.cc:304] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:58.587172  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beb78494b2e0410ba8b06a30fff7158a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:58.587471  3552 ts_tablet_manager.cc:1434] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:13:58.587771  3431 raft_consensus.cc:3060] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.588508  3552 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:58.591970  3551 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.592378  3660 raft_consensus.cc:2749] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.592588  3551 ts_tablet_manager.cc:1403] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.098s	user 0.014s	sys 0.011s
I20260709 13:13:58.593713  3629 leader_election.cc:290] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:58.595288  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a468aba2a1664f7b8a5e1755775a2e72" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:58.595871  3282 raft_consensus.cc:3060] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.595760  3551 raft_consensus.cc:359] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.596943  3551 raft_consensus.cc:385] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.597180  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a468aba2a1664f7b8a5e1755775a2e72" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:58.597934  3357 raft_consensus.cc:3060] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.603749  3552 tablet_bootstrap.cc:654] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.606501  3559 consensus_queue.cc:1048] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.615353  3282 raft_consensus.cc:2468] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:58.620769  3471 leader_election.cc:304] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:58.621640  3629 raft_consensus.cc:2804] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.622117  3358 raft_consensus.cc:2468] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:58.622277  3629 raft_consensus.cc:697] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:13:58.623720  3470 leader_election.cc:304] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:58.623554  3629 consensus_queue.cc:237] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.627054  3584 raft_consensus.cc:2804] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.629503  3554 tablet_bootstrap.cc:654] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.631486  3559 consensus_queue.cc:1048] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:58.627748  3584 raft_consensus.cc:697] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:13:58.634758  3431 raft_consensus.cc:2468] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:58.635385  3584 consensus_queue.cc:237] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.649452  3668 consensus_queue.cc:1048] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.597317  3551 raft_consensus.cc:740] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.653491  3357 raft_consensus.cc:2468] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:58.652278  3551 consensus_queue.cc:260] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
W20260709 13:13:58.659725  3249 consensus_peers.cc:597] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 -> Peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Couldn't send request to peer 73143d6f16c94a3aaef99f8e07588c1f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:58.664379  3551 ts_tablet_manager.cc:1434] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.000s
I20260709 13:13:58.665336  3551 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.670990  3551 tablet_bootstrap.cc:654] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.674165  3668 consensus_queue.cc:1048] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.677736  3552 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:58.678359  3552 ts_tablet_manager.cc:1403] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.090s	user 0.018s	sys 0.042s
I20260709 13:13:58.681901  3660 raft_consensus.cc:493] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.682489  3660 raft_consensus.cc:515] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.685043  3660 leader_election.cc:290] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:58.688004  3552 raft_consensus.cc:359] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.688548  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:58.689291  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:13:58.690356  3398 leader_election.cc:343] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:58.690941  3397 leader_election.cc:343] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.691360  3397 leader_election.cc:304] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:58.692214  3660 raft_consensus.cc:2749] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.688755  3552 raft_consensus.cc:385] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.695622  3552 raft_consensus.cc:740] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.696403  3552 consensus_queue.cc:260] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.695664  3181 catalog_manager.cc:5697] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.698007  3181 catalog_manager.cc:5697] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.700284  3552 ts_tablet_manager.cc:1434] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.001s
I20260709 13:13:58.702790  3552 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:58.712532  3606 consensus_queue.cc:1048] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.712955  3552 tablet_bootstrap.cc:654] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.738138  3553 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:58.738631  3553 ts_tablet_manager.cc:1403] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.231s	user 0.018s	sys 0.028s
I20260709 13:13:58.740020  3554 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:58.740597  3554 ts_tablet_manager.cc:1403] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.179s	user 0.011s	sys 0.000s
I20260709 13:13:58.741261  3553 raft_consensus.cc:359] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.741945  3553 raft_consensus.cc:385] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.744319  3551 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.744769  3551 ts_tablet_manager.cc:1403] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.080s	user 0.016s	sys 0.014s
I20260709 13:13:58.744961  3660 consensus_queue.cc:1048] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.749686  3551 raft_consensus.cc:359] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.751390  3551 raft_consensus.cc:385] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.751742  3551 raft_consensus.cc:740] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.752104  3653 consensus_queue.cc:1048] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.752502  3551 consensus_queue.cc:260] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.742195  3553 raft_consensus.cc:740] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.754981  3551 ts_tablet_manager.cc:1434] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:13:58.755401  3553 consensus_queue.cc:260] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.755995  3551 tablet_bootstrap.cc:492] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.750619  3554 raft_consensus.cc:359] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.756999  3554 raft_consensus.cc:385] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.757324  3554 raft_consensus.cc:740] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.757970  3554 consensus_queue.cc:260] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.759411  3553 ts_tablet_manager.cc:1434] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:13:58.760385  3553 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:58.763110  3551 tablet_bootstrap.cc:654] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.763731  3558 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.764292  3558 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.766657  3558 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:58.768266  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.768759  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:58.769445  3553 tablet_bootstrap.cc:654] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.769464  3431 raft_consensus.cc:2468] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
W20260709 13:13:58.769927  3247 leader_election.cc:343] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.771073  3247 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:58.783411  3551 tablet_bootstrap.cc:492] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.783782  3552 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:58.772178  3558 raft_consensus.cc:2804] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.784145  3551 ts_tablet_manager.cc:1403] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.008s
I20260709 13:13:58.784734  3558 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.785296  3558 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.784319  3552 ts_tablet_manager.cc:1403] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.008s
I20260709 13:13:58.788391  3551 raft_consensus.cc:359] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.789522  3551 raft_consensus.cc:385] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.790202  3564 raft_consensus.cc:493] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.790282  3551 raft_consensus.cc:740] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.791106  3564 raft_consensus.cc:515] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.791671  3551 consensus_queue.cc:260] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.821377  3551 ts_tablet_manager.cc:1434] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.019s
I20260709 13:13:58.822544  3551 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.827302  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e6844fd8644c6498d5b9ce0e7cbd6b" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:58.828217  3281 raft_consensus.cc:2468] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:58.829352  3551 tablet_bootstrap.cc:654] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.830183  3471 leader_election.cc:304] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:58.831446  3629 raft_consensus.cc:2804] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.831934  3629 raft_consensus.cc:493] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.832409  3629 raft_consensus.cc:3060] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.833214  3552 raft_consensus.cc:359] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.834234  3552 raft_consensus.cc:385] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.834618  3552 raft_consensus.cc:740] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.844133  3551 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.844765  3551 ts_tablet_manager.cc:1403] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 13:13:58.846166  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e6844fd8644c6498d5b9ce0e7cbd6b" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:58.847349  3357 raft_consensus.cc:2468] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:58.849128  3620 consensus_queue.cc:1048] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.852289  3559 consensus_queue.cc:1048] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.835618  3552 consensus_queue.cc:260] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.858505  3551 raft_consensus.cc:359] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.862507  3551 raft_consensus.cc:385] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.862054  3552 ts_tablet_manager.cc:1434] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.003s
I20260709 13:13:58.867422  3552 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:58.872200  3551 raft_consensus.cc:740] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.873143  3551 consensus_queue.cc:260] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:58.873940  3564 leader_election.cc:290] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:58.875645  3551 ts_tablet_manager.cc:1434] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.002s
I20260709 13:13:58.876788  3551 tablet_bootstrap.cc:492] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.879427  3554 ts_tablet_manager.cc:1434] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.138s	user 0.006s	sys 0.000s
I20260709 13:13:58.880854  3554 tablet_bootstrap.cc:492] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:58.882480  3552 tablet_bootstrap.cc:654] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.889024  3551 tablet_bootstrap.cc:654] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.893364  3552 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:58.893839  3552 ts_tablet_manager.cc:1403] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 13:13:58.896481  3552 raft_consensus.cc:359] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.897104  3552 raft_consensus.cc:385] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.897347  3552 raft_consensus.cc:740] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.898048  3552 consensus_queue.cc:260] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.900939  3552 ts_tablet_manager.cc:1434] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:13:58.901963  3552 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:58.909592  3551 tablet_bootstrap.cc:492] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.910166  3551 ts_tablet_manager.cc:1403] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.002s
I20260709 13:13:58.912402  3551 raft_consensus.cc:359] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.912900  3551 raft_consensus.cc:385] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.913125  3551 raft_consensus.cc:740] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.913713  3551 consensus_queue.cc:260] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.914939  3551 ts_tablet_manager.cc:1434] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:13:58.915822  3551 tablet_bootstrap.cc:492] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:58.922183  3552 tablet_bootstrap.cc:654] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.927165  3551 tablet_bootstrap.cc:654] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.930164  3620 consensus_queue.cc:1048] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:58.935498  3551 tablet_bootstrap.cc:492] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:58.936187  3551 ts_tablet_manager.cc:1403] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.000s
I20260709 13:13:58.897526  3629 raft_consensus.cc:515] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.942263  3551 raft_consensus.cc:359] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.943002  3551 raft_consensus.cc:385] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.943282  3551 raft_consensus.cc:740] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.943348  3620 consensus_queue.cc:1048] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.944061  3551 consensus_queue.cc:260] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:58.946389  3551 ts_tablet_manager.cc:1434] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:13:58.953987  3551 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
W20260709 13:13:58.954892  3710 log.cc:927] Time spent T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:13:58.963281  3607 raft_consensus.cc:493] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.963835  3607 raft_consensus.cc:515] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:58.965999  3607 leader_election.cc:290] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:58.966609  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98065e64beff449b8caef22a3f2a49f5" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:58.968897  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98065e64beff449b8caef22a3f2a49f5" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:13:58.970952  3249 leader_election.cc:343] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:58.972230  3247 leader_election.cc:343] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.972906  3247 leader_election.cc:304] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:58.974264  3564 raft_consensus.cc:493] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.974972  3564 raft_consensus.cc:515] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:58.975618  3620 raft_consensus.cc:2749] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.988318  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:58.988675  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.001358  3620 raft_consensus.cc:493] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.999567  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e6844fd8644c6498d5b9ce0e7cbd6b" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:59.003836  3471 leader_election.cc:304] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:59.004375  3282 raft_consensus.cc:3060] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.004834  3683 raft_consensus.cc:2749] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.016999  3282 raft_consensus.cc:2468] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:59.018546  3471 leader_election.cc:304] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:59.019526  3683 raft_consensus.cc:2804] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.020145  3683 raft_consensus.cc:697] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:13:59.017832  3620 raft_consensus.cc:515] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.021001  3564 leader_election.cc:290] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:59.021075  3683 consensus_queue.cc:237] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.023123  3553 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:59.023617  3553 ts_tablet_manager.cc:1403] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.263s	user 0.084s	sys 0.136s
I20260709 13:13:59.026328  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.026911  3629 leader_election.cc:290] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.003019  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e6844fd8644c6498d5b9ce0e7cbd6b" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:59.027751  3358 raft_consensus.cc:3060] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:59.028257  3249 leader_election.cc:343] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.027657  3620 leader_election.cc:290] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.030354  3554 tablet_bootstrap.cc:654] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.033970  3551 tablet_bootstrap.cc:654] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.035631  3358 raft_consensus.cc:2468] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:59.036115  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.036818  3432 raft_consensus.cc:2468] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:59.026259  3553 raft_consensus.cc:359] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.038053  3247 leader_election.cc:304] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:59.038362  3553 raft_consensus.cc:385] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.038694  3553 raft_consensus.cc:740] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.039364  3553 consensus_queue.cc:260] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.041445  3553 ts_tablet_manager.cc:1434] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:13:59.042613  3553 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:59.046955  3554 tablet_bootstrap.cc:492] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:59.047458  3554 ts_tablet_manager.cc:1403] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.000s
I20260709 13:13:59.048719  3553 tablet_bootstrap.cc:654] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.049093  3606 raft_consensus.cc:493] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.049619  3606 raft_consensus.cc:515] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.048182  3177 catalog_manager.cc:5697] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.051766  3606 leader_election.cc:290] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.052290  3552 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:59.052104  3554 raft_consensus.cc:359] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.052750  3552 ts_tablet_manager.cc:1403] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.151s	user 0.039s	sys 0.040s
I20260709 13:13:59.052775  3554 raft_consensus.cc:385] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.052986  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d732d672e75142f79a290bd7f5ce0840" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.053305  3554 raft_consensus.cc:740] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.054037  3554 consensus_queue.cc:260] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
W20260709 13:13:59.054454  3396 leader_election.cc:343] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.055428  3551 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:59.055485  3564 raft_consensus.cc:493] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.056013  3551 ts_tablet_manager.cc:1403] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.102s	user 0.012s	sys 0.000s
I20260709 13:13:59.056134  3564 raft_consensus.cc:515] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.057174  3552 raft_consensus.cc:359] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.057869  3552 raft_consensus.cc:385] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.058148  3552 raft_consensus.cc:740] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.058764  3552 consensus_queue.cc:260] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.058876  3551 raft_consensus.cc:359] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.056077  3554 ts_tablet_manager.cc:1434] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:13:59.060297  3551 raft_consensus.cc:385] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.060596  3551 raft_consensus.cc:740] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.060998  3554 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:59.060741  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.061273  3551 consensus_queue.cc:260] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:13:59.062858  3470 leader_election.cc:343] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.063383  3551 ts_tablet_manager.cc:1434] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.001s
I20260709 13:13:59.065603  3553 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:59.066071  3553 ts_tablet_manager.cc:1403] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.004s
I20260709 13:13:59.066879  3551 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:59.066895  3564 leader_election.cc:290] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:59.067493  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.068202  3432 raft_consensus.cc:2468] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:13:59.069418  3470 leader_election.cc:304] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.070164  3606 raft_consensus.cc:493] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.070752  3606 raft_consensus.cc:515] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.073567  3606 leader_election.cc:290] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:59.073460  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.072667  3553 raft_consensus.cc:359] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.074553  3553 raft_consensus.cc:385] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.074821  3553 raft_consensus.cc:740] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
W20260709 13:13:59.075510  3398 leader_election.cc:343] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.075613  3553 consensus_queue.cc:260] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.073164  3564 raft_consensus.cc:2804] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.077606  3564 raft_consensus.cc:493] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.078047  3553 ts_tablet_manager.cc:1434] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:13:59.079066  3553 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:59.078008  3564 raft_consensus.cc:3060] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.075701  3551 tablet_bootstrap.cc:654] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.084794  3553 tablet_bootstrap.cc:654] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.089044  3721 raft_consensus.cc:2804] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.089646  3721 raft_consensus.cc:493] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.092090  3551 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:59.090179  3721 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.093681  3551 ts_tablet_manager.cc:1403] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 13:13:59.097180  3552 ts_tablet_manager.cc:1434] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.044s	user 0.016s	sys 0.012s
I20260709 13:13:59.097368  3606 raft_consensus.cc:493] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.098198  3552 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:59.097857  3606 raft_consensus.cc:515] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.099279  3551 raft_consensus.cc:359] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.099959  3551 raft_consensus.cc:385] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.100241  3551 raft_consensus.cc:740] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.100871  3551 consensus_queue.cc:260] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.102726  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:13:59.104242  3396 leader_election.cc:343] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.112936  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:59.114801  3398 leader_election.cc:343] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.115379  3398 leader_election.cc:304] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.121806  3564 raft_consensus.cc:515] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.128405  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.129592  3559 raft_consensus.cc:493] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.129834  3564 leader_election.cc:290] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:59.130107  3559 raft_consensus.cc:515] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.131707  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:59.132174  3668 raft_consensus.cc:493] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.132345  3430 raft_consensus.cc:3060] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.132786  3668 raft_consensus.cc:515] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.133728  3683 consensus_queue.cc:1048] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:59.134895  3668 leader_election.cc:290] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:59.127010  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d732d672e75142f79a290bd7f5ce0840" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.128082  3553 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:59.136010  3282 raft_consensus.cc:2468] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:59.132068  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:59.136215  3553 ts_tablet_manager.cc:1403] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.057s	user 0.019s	sys 0.019s
I20260709 13:13:59.137238  3552 tablet_bootstrap.cc:654] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:59.138329  3470 leader_election.cc:343] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.138470  3629 consensus_queue.cc:1048] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:59.140564  3559 leader_election.cc:290] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.141407  3553 raft_consensus.cc:359] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.142136  3553 raft_consensus.cc:385] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.142627  3553 raft_consensus.cc:740] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.142671  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.143541  3358 raft_consensus.cc:2468] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:13:59.143492  3553 consensus_queue.cc:260] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.145467  3396 leader_election.cc:304] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:59.147593  3653 raft_consensus.cc:2804] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.147980  3653 raft_consensus.cc:493] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.148341  3653 raft_consensus.cc:3060] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.148653  3553 ts_tablet_manager.cc:1434] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:13:59.148639  3660 raft_consensus.cc:2749] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.150094  3553 tablet_bootstrap.cc:492] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:59.155097  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.156352  3606 leader_election.cc:290] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
W20260709 13:13:59.158008  3397 leader_election.cc:343] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.158818  3553 tablet_bootstrap.cc:654] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.158636  3397 leader_election.cc:304] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:59.160550  3397 leader_election.cc:304] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.161916  3559 raft_consensus.cc:2749] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.162724  3559 raft_consensus.cc:2804] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.163141  3559 raft_consensus.cc:493] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.163717  3559 raft_consensus.cc:3060] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.127825  3551 ts_tablet_manager.cc:1434] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 13:13:59.169279  3551 tablet_bootstrap.cc:492] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:59.171017  3553 tablet_bootstrap.cc:492] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:59.171567  3553 ts_tablet_manager.cc:1403] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 13:13:59.002336  3558 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:13:59.185132  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 90854us
I20260709 13:13:59.189821  3552 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:59.191712  3552 ts_tablet_manager.cc:1403] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.092s	user 0.017s	sys 0.012s
I20260709 13:13:59.193344  3553 raft_consensus.cc:359] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.194311  3553 raft_consensus.cc:385] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.194653  3553 raft_consensus.cc:740] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.197746  3553 consensus_queue.cc:260] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.180673  3430 raft_consensus.cc:2468] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:13:59.209371  3553 ts_tablet_manager.cc:1434] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.037s	user 0.002s	sys 0.005s
I20260709 13:13:59.210999  3668 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.211107  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.211591  3668 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.215143  3653 raft_consensus.cc:515] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.215427  3668 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:59.216410  3431 raft_consensus.cc:2393] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 06964934536148c2aa77a2f1c6382fea in this term.
I20260709 13:13:59.217271  3653 leader_election.cc:290] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.218286  3721 raft_consensus.cc:515] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.219178  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:59.219780  3356 raft_consensus.cc:3060] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.220759  3470 leader_election.cc:304] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.221771  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:59.221966  3564 raft_consensus.cc:2804] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.222533  3564 raft_consensus.cc:697] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
W20260709 13:13:59.222997  3323 leader_election.cc:343] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.223335  3564 consensus_queue.cc:237] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
W20260709 13:13:59.224215  3397 leader_election.cc:343] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.226225  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:59.226874  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.228008  3322 leader_election.cc:304] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
W20260709 13:13:59.228612  3249 leader_election.cc:343] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.231000  3551 tablet_bootstrap.cc:654] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.232527  3553 tablet_bootstrap.cc:492] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:59.222195  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.237257  3552 raft_consensus.cc:359] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.237906  3552 raft_consensus.cc:385] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.238090  3552 raft_consensus.cc:740] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.238574  3552 consensus_queue.cc:260] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.239943  3729 raft_consensus.cc:493] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.240465  3729 raft_consensus.cc:515] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.242659  3729 leader_election.cc:290] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.244755  3431 raft_consensus.cc:2468] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:13:59.247109  3553 tablet_bootstrap.cc:654] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.248332  3653 raft_consensus.cc:493] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.248831  3653 raft_consensus.cc:515] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.232695  3721 leader_election.cc:290] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.252794  3322 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:59.253674  3623 raft_consensus.cc:2804] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.254047  3623 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.254376  3623 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.255144  3683 consensus_queue.cc:1048] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:59.248912  3356 raft_consensus.cc:2468] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:59.262290  3653 leader_election.cc:290] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.264441  3355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.267379  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.268122  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.269496  3683 consensus_queue.cc:1048] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:13:59.287407  3396 leader_election.cc:343] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.270684  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.289175  3357 raft_consensus.cc:2468] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
W20260709 13:13:59.290174  3249 leader_election.cc:343] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:59.271529  3397 leader_election.cc:343] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.271977  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.278285  3668 raft_consensus.cc:2749] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.294036  3282 raft_consensus.cc:2393] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:13:59.295931  3554 tablet_bootstrap.cc:654] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:59.293064  3398 leader_election.cc:343] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.296962  3398 leader_election.cc:304] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.298309  3247 leader_election.cc:304] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:59.299399  3653 raft_consensus.cc:2749] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.301829  3723 raft_consensus.cc:2804] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.302251  3723 raft_consensus.cc:493] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.302668  3723 raft_consensus.cc:3060] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.288483  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:59.312430  3431 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.332293  3558 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:59.332996  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:59.333386  3355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:59.333841  3432 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.293036  3559 raft_consensus.cc:515] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.336791  3396 leader_election.cc:304] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:13:59.337949  3653 raft_consensus.cc:2804] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.338266  3559 leader_election.cc:290] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.338456  3653 raft_consensus.cc:697] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:13:59.339316  3653 consensus_queue.cc:237] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.364529  3179 catalog_manager.cc:5697] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.354940  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d732d672e75142f79a290bd7f5ce0840" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:59.344384  3623 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.380254  3552 ts_tablet_manager.cc:1434] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.188s	user 0.017s	sys 0.048s
I20260709 13:13:59.381250  3552 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:59.381405  3506 raft_consensus.cc:3060] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.376330  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d732d672e75142f79a290bd7f5ce0840" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:59.383186  3283 raft_consensus.cc:3060] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.374185  3606 raft_consensus.cc:493] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.387308  3606 raft_consensus.cc:515] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.389374  3552 tablet_bootstrap.cc:654] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.389463  3606 leader_election.cc:290] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.391300  3354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49acd4ec7dc42a8b9a1cb2835d1bede" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.374761  3356 raft_consensus.cc:3060] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:59.415457  3396 leader_election.cc:343] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.416467  3355 raft_consensus.cc:2393] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 972a4469a4654416ab0a153d8c740282 in current term 1: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
I20260709 13:13:59.415910  3553 tablet_bootstrap.cc:492] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:59.419036  3553 ts_tablet_manager.cc:1403] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.209s	user 0.017s	sys 0.045s
I20260709 13:13:59.420693  3653 raft_consensus.cc:493] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.426658  3653 raft_consensus.cc:515] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.429500  3653 leader_election.cc:290] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.429716  3355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.430064  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:13:59.430971  3396 leader_election.cc:343] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.434293  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49acd4ec7dc42a8b9a1cb2835d1bede" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:59.435531  3398 leader_election.cc:343] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.435933  3398 leader_election.cc:304] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.436444  3432 raft_consensus.cc:2468] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:59.437703  3660 raft_consensus.cc:2749] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.438732  3247 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.438961  3721 raft_consensus.cc:493] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.439302  3623 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:59.412851  3723 raft_consensus.cc:515] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.423525  3283 raft_consensus.cc:2468] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:59.440321  3733 raft_consensus.cc:2804] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:13:59.436549  3397 leader_election.cc:343] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.444447  3397 leader_election.cc:304] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:59.443897  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:13:59.444805  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:59.445747  3357 raft_consensus.cc:3060] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.446151  3559 raft_consensus.cc:2749] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.439498  3721 raft_consensus.cc:515] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.448503  3721 leader_election.cc:290] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.449191  3553 raft_consensus.cc:359] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.449841  3553 raft_consensus.cc:385] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.450105  3553 raft_consensus.cc:740] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.440864  3733 raft_consensus.cc:697] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:59.442770  3723 leader_election.cc:290] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.445695  3397 leader_election.cc:304] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:59.452689  3733 consensus_queue.cc:237] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.453094  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:59.453965  3430 raft_consensus.cc:2393] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:13:59.454182  3584 consensus_queue.cc:1048] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:59.453024  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:59.455621  3249 leader_election.cc:343] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.456105  3606 raft_consensus.cc:2804] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.424679  3431 raft_consensus.cc:2468] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:59.456632  3606 raft_consensus.cc:697] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:13:59.459492  3357 raft_consensus.cc:2468] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:59.462327  3247 leader_election.cc:304] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:59.457480  3606 consensus_queue.cc:237] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.464030  3553 consensus_queue.cc:260] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.464926  3723 raft_consensus.cc:2804] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.477772  3553 ts_tablet_manager.cc:1434] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 13:13:59.479205  3553 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:59.480809  3667 raft_consensus.cc:493] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.481278  3667 raft_consensus.cc:515] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.483330  3667 leader_election.cc:290] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.471644  3723 raft_consensus.cc:697] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:59.478595  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.477155  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:59.491272  3432 raft_consensus.cc:2468] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:13:59.489914  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.492637  3431 raft_consensus.cc:2468] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:13:59.494190  3554 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:59.494685  3554 ts_tablet_manager.cc:1403] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.434s	user 0.021s	sys 0.004s
I20260709 13:13:59.479125  3247 leader_election.cc:304] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:59.496920  3322 leader_election.cc:304] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:13:59.497768  3179 catalog_manager.cc:5697] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.500147  3552 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:59.500249  3721 raft_consensus.cc:2804] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.500710  3552 ts_tablet_manager.cc:1403] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.120s	user 0.014s	sys 0.010s
I20260709 13:13:59.500780  3721 raft_consensus.cc:697] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:59.501739  3721 consensus_queue.cc:237] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.503507  3552 raft_consensus.cc:359] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.505265  3552 raft_consensus.cc:385] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.468600  3356 raft_consensus.cc:2468] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:13:59.505550  3552 raft_consensus.cc:740] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.509511  3552 consensus_queue.cc:260] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.508315  3553 tablet_bootstrap.cc:654] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.514614  3552 ts_tablet_manager.cc:1434] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:13:59.515945  3552 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
W20260709 13:13:59.514467  3249 leader_election.cc:343] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.502087  3723 consensus_queue.cc:237] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.522284  3247 leader_election.cc:304] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:59.508203  3667 raft_consensus.cc:2804] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.512499  3653 raft_consensus.cc:493] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.523993  3554 raft_consensus.cc:359] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.530282  3554 raft_consensus.cc:385] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.530570  3554 raft_consensus.cc:740] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.528977  3551 tablet_bootstrap.cc:492] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:59.532944  3551 ts_tablet_manager.cc:1403] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.364s	user 0.005s	sys 0.019s
I20260709 13:13:59.531330  3554 consensus_queue.cc:260] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.524664  3323 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:59.524205  3558 raft_consensus.cc:2804] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.538043  3558 raft_consensus.cc:493] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.538378  3558 raft_consensus.cc:3060] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.540009  3623 raft_consensus.cc:2749] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.524026  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.524827  3667 raft_consensus.cc:493] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.542338  3667 raft_consensus.cc:3060] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:59.542497  3324 leader_election.cc:343] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.525435  3653 raft_consensus.cc:515] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.548285  3551 raft_consensus.cc:359] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
W20260709 13:13:59.548900  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:59.550237  3653 leader_election.cc:290] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:59.551271  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:13:59.552752  3397 leader_election.cc:343] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.554432  3584 raft_consensus.cc:493] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.554936  3584 raft_consensus.cc:515] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.557917  3584 leader_election.cc:290] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:59.562750  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.548945  3551 raft_consensus.cc:385] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.563907  3551 raft_consensus.cc:740] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.567693  3554 ts_tablet_manager.cc:1434] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.073s	user 0.004s	sys 0.004s
I20260709 13:13:59.572242  3551 consensus_queue.cc:260] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.571347  3179 catalog_manager.cc:5697] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:59.569248  3398 leader_election.cc:343] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.572000  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.581949  3554 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:59.582619  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.583410  3432 raft_consensus.cc:2393] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:13:59.584936  3551 ts_tablet_manager.cc:1434] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 13:13:59.585597  3551 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:59.590332  3552 tablet_bootstrap.cc:654] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.586302  3398 leader_election.cc:304] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:59.595145  3653 raft_consensus.cc:2749] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.596035  3554 tablet_bootstrap.cc:654] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.595964  3470 leader_election.cc:304] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:59.596990  3683 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.602535  3623 raft_consensus.cc:493] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.603118  3584 consensus_queue.cc:1048] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:59.603030  3623 raft_consensus.cc:515] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.611614  3623 leader_election.cc:290] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.613894  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.614683  3432 raft_consensus.cc:2393] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:13:59.615556  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.617997  3683 raft_consensus.cc:2749] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.622929  3322 leader_election.cc:304] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:59.609109  3667 raft_consensus.cc:515] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.599716  3558 raft_consensus.cc:515] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.629582  3551 tablet_bootstrap.cc:654] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.630895  3623 raft_consensus.cc:3060] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.647852  3554 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:59.648094  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:59.648406  3554 ts_tablet_manager.cc:1403] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.013s
I20260709 13:13:59.648725  3432 raft_consensus.cc:3060] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.651177  3554 raft_consensus.cc:359] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.651875  3554 raft_consensus.cc:385] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.652163  3554 raft_consensus.cc:740] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.652642  3558 leader_election.cc:290] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.652793  3554 consensus_queue.cc:260] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.654098  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:59.654707  3506 raft_consensus.cc:3060] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.654896  3554 ts_tablet_manager.cc:1434] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:13:59.655830  3554 tablet_bootstrap.cc:492] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:59.656317  3179 catalog_manager.cc:5697] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.657482  3667 leader_election.cc:290] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.658667  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:59.658100  3179 catalog_manager.cc:5697] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.659031  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:59.659219  3431 raft_consensus.cc:3060] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.660113  3623 raft_consensus.cc:2749] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:13:59.661011  3324 leader_election.cc:343] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.661418  3432 raft_consensus.cc:2468] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:59.662532  3247 leader_election.cc:304] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:59.663692  3558 raft_consensus.cc:2804] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.664211  3558 raft_consensus.cc:697] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:59.665021  3558 consensus_queue.cc:237] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.667224  3431 raft_consensus.cc:2468] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:13:59.668726  3322 leader_election.cc:304] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:59.670023  3623 raft_consensus.cc:2804] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.670814  3623 raft_consensus.cc:697] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:13:59.671774  3623 consensus_queue.cc:237] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.675663  3553 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:59.676363  3553 ts_tablet_manager.cc:1403] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.197s	user 0.012s	sys 0.003s
I20260709 13:13:59.678423  3552 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:59.678881  3554 tablet_bootstrap.cc:654] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.679834  3553 raft_consensus.cc:359] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.682847  3553 raft_consensus.cc:385] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.683208  3553 raft_consensus.cc:740] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.684082  3553 consensus_queue.cc:260] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.679852  3506 raft_consensus.cc:2468] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:59.678906  3552 ts_tablet_manager.cc:1403] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.163s	user 0.006s	sys 0.006s
I20260709 13:13:59.701828  3553 ts_tablet_manager.cc:1434] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:13:59.704072  3553 tablet_bootstrap.cc:492] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:59.703960  3552 raft_consensus.cc:359] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.704820  3552 raft_consensus.cc:385] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.705152  3552 raft_consensus.cc:740] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.705914  3552 consensus_queue.cc:260] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.708248  3552 ts_tablet_manager.cc:1434] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:13:59.709089  3552 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:59.712478  3721 raft_consensus.cc:493] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.712952  3721 raft_consensus.cc:515] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.713702  3553 tablet_bootstrap.cc:654] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.715061  3721 leader_election.cc:290] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:59.718466  3552 tablet_bootstrap.cc:654] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.718664  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.719420  3358 raft_consensus.cc:2393] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 972a4469a4654416ab0a153d8c740282 in current term 1: Already voted for candidate 06964934536148c2aa77a2f1c6382fea in this term.
I20260709 13:13:59.719681  3551 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:59.720166  3551 ts_tablet_manager.cc:1403] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.135s	user 0.011s	sys 0.011s
I20260709 13:13:59.720952  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.722383  3247 leader_election.cc:304] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972a4469a4654416ab0a153d8c740282; no voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.722983  3551 raft_consensus.cc:359] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.723639  3551 raft_consensus.cc:385] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.723887  3551 raft_consensus.cc:740] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.724630  3551 consensus_queue.cc:260] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.726720  3551 ts_tablet_manager.cc:1434] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:13:59.727624  3551 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:59.728188  3721 raft_consensus.cc:3060] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.738562  3554 tablet_bootstrap.cc:492] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:59.739167  3554 ts_tablet_manager.cc:1403] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.003s
I20260709 13:13:59.739431  3733 raft_consensus.cc:493] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.739923  3733 raft_consensus.cc:515] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.742062  3733 leader_election.cc:290] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.742930  3177 catalog_manager.cc:5697] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.744115  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.748594  3552 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:59.749118  3552 ts_tablet_manager.cc:1403] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.003s
I20260709 13:13:59.745519  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:13:59.753657  3431 raft_consensus.cc:2468] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
W20260709 13:13:59.749214  3249 leader_election.cc:343] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.749527  3178 catalog_manager.cc:5697] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.755926  3552 raft_consensus.cc:359] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.755775  3178 catalog_manager.cc:5697] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.756639  3552 raft_consensus.cc:385] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.742022  3554 raft_consensus.cc:359] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.756942  3551 tablet_bootstrap.cc:654] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.757193  3554 raft_consensus.cc:385] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.757493  3554 raft_consensus.cc:740] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.758105  3554 consensus_queue.cc:260] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.752501  3721 raft_consensus.cc:2749] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.758474  3247 leader_election.cc:304] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:59.756968  3552 raft_consensus.cc:740] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.760423  3554 ts_tablet_manager.cc:1434] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:13:59.760869  3721 raft_consensus.cc:2804] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.760785  3552 consensus_queue.cc:260] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.761415  3554 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:59.761363  3721 raft_consensus.cc:493] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.762259  3721 raft_consensus.cc:3060] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.763070  3552 ts_tablet_manager.cc:1434] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:13:59.763492  3623 raft_consensus.cc:493] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.763978  3623 raft_consensus.cc:515] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.766041  3623 leader_election.cc:290] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:59.766098  3558 raft_consensus.cc:493] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.766788  3558 raft_consensus.cc:515] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.767560  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98065e64beff449b8caef22a3f2a49f5" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.768220  3282 raft_consensus.cc:2468] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:13:59.769249  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98065e64beff449b8caef22a3f2a49f5" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:13:59.769367  3558 leader_election.cc:290] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.770583  3323 leader_election.cc:304] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:59.771528  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59159d79fbfc4835a64e52a2783f46f7" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:13:59.772624  3324 leader_election.cc:343] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:59.773898  3249 leader_election.cc:343] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.776870  3623 raft_consensus.cc:2804] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:59.784209  3623 raft_consensus.cc:493] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.777586  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59159d79fbfc4835a64e52a2783f46f7" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:13:59.784622  3623 raft_consensus.cc:3060] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:59.785737  3247 leader_election.cc:343] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.786206  3247 leader_election.cc:304] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:13:59.787276  3558 raft_consensus.cc:2749] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.764394  3552 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:59.780375  3553 tablet_bootstrap.cc:492] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:59.788568  3553 ts_tablet_manager.cc:1403] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.005s
I20260709 13:13:59.791419  3551 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:59.791338  3553 raft_consensus.cc:359] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.792147  3553 raft_consensus.cc:385] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.792434  3553 raft_consensus.cc:740] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.793187  3553 consensus_queue.cc:260] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.794016  3551 ts_tablet_manager.cc:1403] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.004s
I20260709 13:13:59.795158  3552 tablet_bootstrap.cc:654] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.795437  3553 ts_tablet_manager.cc:1434] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:13:59.796434  3553 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:13:59.796710  3551 raft_consensus.cc:359] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.797395  3551 raft_consensus.cc:385] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.797668  3551 raft_consensus.cc:740] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.798338  3551 consensus_queue.cc:260] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.803416  3623 raft_consensus.cc:515] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.806056  3623 leader_election.cc:290] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:13:59.808277  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98065e64beff449b8caef22a3f2a49f5" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:59.809943  3554 tablet_bootstrap.cc:654] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.810500  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98065e64beff449b8caef22a3f2a49f5" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:13:59.811086  3282 raft_consensus.cc:3060] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.782158  3721 raft_consensus.cc:515] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
W20260709 13:13:59.815457  3324 leader_election.cc:343] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.818102  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:13:59.818658  3431 raft_consensus.cc:3060] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:59.818851  3551 ts_tablet_manager.cc:1434] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 13:13:59.819442  3683 consensus_queue.cc:1048] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:59.824486  3551 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:13:59.835901  3551 tablet_bootstrap.cc:654] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.843349  3683 consensus_queue.cc:1048] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:59.851111  3558 consensus_queue.cc:1048] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:59.826958  3431 raft_consensus.cc:2468] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:13:59.878136  3282 raft_consensus.cc:2468] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:13:59.875635  3721 leader_election.cc:290] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:13:59.880841  3660 raft_consensus.cc:493] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.881523  3553 tablet_bootstrap.cc:654] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.881330  3660 raft_consensus.cc:515] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:13:59.881861  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:13:59.883486  3660 leader_election.cc:290] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:13:59.879766  3323 leader_election.cc:304] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:13:59.884989  3623 raft_consensus.cc:2804] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.885558  3623 raft_consensus.cc:697] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:13:59.886407  3623 consensus_queue.cc:237] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.890049  3247 leader_election.cc:304] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:13:59.902451  3551 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:13:59.903180  3551 ts_tablet_manager.cc:1403] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.018s
I20260709 13:13:59.893242  3721 raft_consensus.cc:2804] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:59.904793  3721 raft_consensus.cc:697] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:13:59.905323  3554 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:59.906002  3554 ts_tablet_manager.cc:1403] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.145s	user 0.026s	sys 0.026s
I20260709 13:13:59.905917  3721 consensus_queue.cc:237] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.890504  3606 consensus_queue.cc:1048] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:13:59.891173  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6db7a8007940d6b8bc088af66589ae" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:13:59.893255  3249 leader_election.cc:343] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.918154  3554 raft_consensus.cc:359] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.918947  3554 raft_consensus.cc:385] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.919243  3554 raft_consensus.cc:740] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.919369  3552 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:13:59.920780  3552 ts_tablet_manager.cc:1403] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.157s	user 0.013s	sys 0.038s
I20260709 13:13:59.908790  3683 raft_consensus.cc:493] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.920008  3554 consensus_queue.cc:260] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.928335  3554 ts_tablet_manager.cc:1434] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.003s
I20260709 13:13:59.920881  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6db7a8007940d6b8bc088af66589ae" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:13:59.929698  3551 raft_consensus.cc:359] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
W20260709 13:13:59.930229  3396 leader_election.cc:343] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.932940  3554 tablet_bootstrap.cc:492] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:13:59.933458  3552 raft_consensus.cc:359] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.931449  3551 raft_consensus.cc:385] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.934178  3552 raft_consensus.cc:385] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.934376  3551 raft_consensus.cc:740] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.934559  3552 raft_consensus.cc:740] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.935187  3551 consensus_queue.cc:260] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.937451  3551 ts_tablet_manager.cc:1434] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.034s	user 0.000s	sys 0.006s
I20260709 13:13:59.938387  3551 tablet_bootstrap.cc:492] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
W20260709 13:13:59.939635  3397 leader_election.cc:343] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:59.939805  3554 tablet_bootstrap.cc:654] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.940181  3397 leader_election.cc:304] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:13:59.940937  3606 raft_consensus.cc:2749] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.935386  3552 consensus_queue.cc:260] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:13:59.956450  3553 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:13:59.958083  3553 ts_tablet_manager.cc:1403] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.162s	user 0.021s	sys 0.008s
I20260709 13:13:59.958272  3552 ts_tablet_manager.cc:1434] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:13:59.958169  3551 tablet_bootstrap.cc:654] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:59.964191  3552 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:13:59.967865  3554 tablet_bootstrap.cc:492] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:13:59.968549  3554 ts_tablet_manager.cc:1403] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.024s
I20260709 13:13:59.924813  3683 raft_consensus.cc:515] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.978071  3683 leader_election.cc:290] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:13:59.979105  3653 consensus_queue.cc:1048] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:59.975835  3553 raft_consensus.cc:359] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.984126  3553 raft_consensus.cc:385] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:59.984476  3553 raft_consensus.cc:740] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:13:59.985209  3553 consensus_queue.cc:260] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:13:59.972633  3606 raft_consensus.cc:493] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.973949  3558 consensus_queue.cc:1048] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:13:59.999629  3554 raft_consensus.cc:359] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.001022  3554 raft_consensus.cc:385] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.001309  3554 raft_consensus.cc:740] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.002152  3553 ts_tablet_manager.cc:1434] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:14:00.003307  3553 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:00.002991  3554 consensus_queue.cc:260] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:13:59.994454  3606 raft_consensus.cc:515] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.010217  3606 leader_election.cc:290] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:00.016551  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596e0222214f40b5861691b7bcaeea24" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.021668  3554 ts_tablet_manager.cc:1434] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:14:00.023011  3554 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:00.027531  3660 raft_consensus.cc:493] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.028101  3660 raft_consensus.cc:515] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
W20260709 13:14:00.030843  3398 leader_election.cc:343] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.031471  3653 consensus_queue.cc:1048] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.031984  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb18f85836cc456b8362bb08424827ca" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:00.032826  3280 raft_consensus.cc:2468] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:14:00.034932  3397 leader_election.cc:304] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:00.035415  3180 catalog_manager.cc:5697] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:00.037024  3653 raft_consensus.cc:2804] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.037449  3653 raft_consensus.cc:493] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.037925  3653 raft_consensus.cc:3060] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.037305  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb18f85836cc456b8362bb08424827ca" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.040848  3660 leader_election.cc:290] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
W20260709 13:14:00.046416  3398 leader_election.cc:343] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.047663  3623 raft_consensus.cc:493] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.048333  3623 raft_consensus.cc:515] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.050901  3623 leader_election.cc:290] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:00.051707  3623 raft_consensus.cc:493] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.052004  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59159d79fbfc4835a64e52a2783f46f7" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:00.023504  3177 catalog_manager.cc:5697] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:00.055461  3653 raft_consensus.cc:515] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.059509  3554 tablet_bootstrap.cc:654] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.060484  3553 tablet_bootstrap.cc:654] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.052713  3280 raft_consensus.cc:2468] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:14:00.060662  3552 tablet_bootstrap.cc:654] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.060199  3733 consensus_queue.cc:1048] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.063086  3323 leader_election.cc:304] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:00.063933  3667 raft_consensus.cc:2804] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.064333  3667 raft_consensus.cc:493] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.064664  3667 raft_consensus.cc:3060] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.052297  3623 raft_consensus.cc:515] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.075533  3623 leader_election.cc:290] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:00.090662  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb18f85836cc456b8362bb08424827ca" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:00.090737  3620 consensus_queue.cc:1048] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:00.091392  3282 raft_consensus.cc:3060] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.092658  3660 raft_consensus.cc:493] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.093137  3660 raft_consensus.cc:515] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.092387  3733 consensus_queue.cc:1048] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.132117  3660 leader_election.cc:290] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:00.133538  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:00.134183  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:14:00.135825  3397 leader_election.cc:343] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.095757  3653 leader_election.cc:290] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:00.096361  3552 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:00.137578  3552 ts_tablet_manager.cc:1403] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.174s	user 0.022s	sys 0.007s
I20260709 13:14:00.141127  3552 raft_consensus.cc:359] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.141816  3552 raft_consensus.cc:385] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.096480  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49acd4ec7dc42a8b9a1cb2835d1bede" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.142755  3504 raft_consensus.cc:2468] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:14:00.143401  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596e0222214f40b5861691b7bcaeea24" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:00.103528  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb18f85836cc456b8362bb08424827ca" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
W20260709 13:14:00.144723  3470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 159364us
I20260709 13:14:00.104569  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49acd4ec7dc42a8b9a1cb2835d1bede" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:00.145947  3429 raft_consensus.cc:2468] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:14:00.107450  3779 consensus_queue.cc:1048] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:00.147208  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:00.151034  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:00.151855  3429 raft_consensus.cc:2393] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
W20260709 13:14:00.154561  3398 leader_election.cc:343] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.155025  3398 leader_election.cc:304] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:00.155833  3324 leader_election.cc:304] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:00.156188  3606 raft_consensus.cc:2749] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:00.156922  3779 raft_consensus.cc:2804] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.157363  3779 raft_consensus.cc:493] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:00.157701  3396 leader_election.cc:343] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.158337  3396 leader_election.cc:304] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:00.142159  3552 raft_consensus.cc:740] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.112990  3667 raft_consensus.cc:515] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.094866  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59159d79fbfc4835a64e52a2783f46f7" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.108709  3553 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:00.165426  3552 consensus_queue.cc:260] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.166242  3553 ts_tablet_manager.cc:1403] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.163s	user 0.015s	sys 0.000s
I20260709 13:14:00.167727  3552 ts_tablet_manager.cc:1434] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:14:00.168896  3552 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
W20260709 13:14:00.169186  3324 leader_election.cc:343] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.169056  3553 raft_consensus.cc:359] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.169600  3282 raft_consensus.cc:2468] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:00.169814  3553 raft_consensus.cc:385] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.170163  3553 raft_consensus.cc:740] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.157760  3779 raft_consensus.cc:3060] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.129144  3551 tablet_bootstrap.cc:492] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:00.159602  3782 raft_consensus.cc:2749] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:00.171672  3553 consensus_queue.cc:260] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.171890  3397 leader_election.cc:304] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:00.172377  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59159d79fbfc4835a64e52a2783f46f7" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:00.174224  3554 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:00.174371  3551 ts_tablet_manager.cc:1403] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.236s	user 0.016s	sys 0.009s
I20260709 13:14:00.177678  3554 ts_tablet_manager.cc:1403] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.155s	user 0.009s	sys 0.033s
I20260709 13:14:00.180560  3553 ts_tablet_manager.cc:1434] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:14:00.181299  3553 tablet_bootstrap.cc:492] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:00.185870  3782 raft_consensus.cc:2804] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:00.186479  3782 raft_consensus.cc:697] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:00.188236  3503 raft_consensus.cc:3060] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.187405  3782 consensus_queue.cc:237] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
W20260709 13:14:00.192207  3324 leader_election.cc:343] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.193701  3779 raft_consensus.cc:515] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.201212  3551 raft_consensus.cc:359] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.201956  3551 raft_consensus.cc:385] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.202225  3551 raft_consensus.cc:740] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.202693  3779 leader_election.cc:290] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:00.202917  3551 consensus_queue.cc:260] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.203253  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59159d79fbfc4835a64e52a2783f46f7" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:00.204304  3283 raft_consensus.cc:3060] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.204789  3554 raft_consensus.cc:359] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.205493  3554 raft_consensus.cc:385] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.205753  3554 raft_consensus.cc:740] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.206348  3554 consensus_queue.cc:260] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.206483  3653 consensus_queue.cc:1048] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.207609  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49acd4ec7dc42a8b9a1cb2835d1bede" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:00.208246  3505 raft_consensus.cc:3060] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.206687  3667 leader_election.cc:290] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
W20260709 13:14:00.212572  3249 consensus_peers.cc:597] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 -> Peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Couldn't send request to peer 73143d6f16c94a3aaef99f8e07588c1f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:00.213964  3774 raft_consensus.cc:493] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.214476  3774 raft_consensus.cc:515] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.214833  3470 leader_election.cc:304] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:00.215286  3553 tablet_bootstrap.cc:654] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.215619  3683 raft_consensus.cc:2749] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:00.216709  3774 leader_election.cc:290] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:00.217238  3667 consensus_queue.cc:1048] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:14:00.178774  3398 leader_election.cc:343] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.221387  3554 ts_tablet_manager.cc:1434] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 13:14:00.218299  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:00.222505  3554 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:00.226121  3721 consensus_queue.cc:1048] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.226439  3283 raft_consensus.cc:2468] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
W20260709 13:14:00.225513  3789 log.cc:927] Time spent T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:14:00.223682  3786 consensus_queue.cc:1048] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:14:00.227690  3323 leader_election.cc:304] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:00.228691  3779 raft_consensus.cc:2804] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:00.220988  3505 raft_consensus.cc:2468] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:00.229159  3779 raft_consensus.cc:697] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:00.231469  3551 ts_tablet_manager.cc:1434] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:14:00.234292  3552 tablet_bootstrap.cc:654] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.237936  3324 leader_election.cc:304] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:00.239988  3551 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:00.241798  3778 raft_consensus.cc:2804] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:00.242449  3778 raft_consensus.cc:697] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:00.246590  3778 consensus_queue.cc:237] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.249874  3551 tablet_bootstrap.cc:654] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.229974  3779 consensus_queue.cc:237] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.273885  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:00.275153  3431 raft_consensus.cc:2393] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:14:00.276474  3470 leader_election.cc:304] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:00.278004  3683 raft_consensus.cc:3060] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:00.261490  3398 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 85156.7us
I20260709 13:14:00.322208  3667 raft_consensus.cc:493] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.324903  3667 raft_consensus.cc:515] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.327086  3667 leader_election.cc:290] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:00.328128  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49acd4ec7dc42a8b9a1cb2835d1bede" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:00.328756  3430 raft_consensus.cc:3060] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.333020  3552 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:00.333666  3552 ts_tablet_manager.cc:1403] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.165s	user 0.016s	sys 0.003s
I20260709 13:14:00.339043  3683 raft_consensus.cc:2749] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:00.341351  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6db7a8007940d6b8bc088af66589ae" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:00.342075  3283 raft_consensus.cc:2468] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:14:00.343182  3323 leader_election.cc:304] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:00.346184  3623 raft_consensus.cc:2804] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.346701  3623 raft_consensus.cc:493] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.347146  3623 raft_consensus.cc:3060] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.349423  3733 consensus_queue.cc:1048] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.358839  3430 raft_consensus.cc:2468] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:00.368088  3553 tablet_bootstrap.cc:492] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:00.368731  3553 ts_tablet_manager.cc:1403] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.188s	user 0.028s	sys 0.004s
I20260709 13:14:00.378014  3667 consensus_queue.cc:1048] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:00.383361  3553 raft_consensus.cc:359] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.384215  3553 raft_consensus.cc:385] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.384483  3553 raft_consensus.cc:740] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.385762  3553 consensus_queue.cc:260] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.390745  3623 raft_consensus.cc:515] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.396886  3553 ts_tablet_manager.cc:1434] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:14:00.397806  3553 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:00.398527  3552 raft_consensus.cc:359] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.399189  3552 raft_consensus.cc:385] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.399410  3552 raft_consensus.cc:740] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.407861  3552 consensus_queue.cc:260] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.406543  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6db7a8007940d6b8bc088af66589ae" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:00.414083  3551 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:00.414515  3551 ts_tablet_manager.cc:1403] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.175s	user 0.011s	sys 0.004s
I20260709 13:14:00.427064  3552 ts_tablet_manager.cc:1434] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.000s
I20260709 13:14:00.427917  3552 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:00.428529  3554 tablet_bootstrap.cc:654] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.434228  3721 consensus_queue.cc:1048] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:00.418320  3429 raft_consensus.cc:2468] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:14:00.459843  3551 raft_consensus.cc:359] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.460846  3551 raft_consensus.cc:385] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.461202  3551 raft_consensus.cc:740] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.465799  3551 consensus_queue.cc:260] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:14:00.464517  3323 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 57949.6us
I20260709 13:14:00.482071  3623 leader_election.cc:290] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:00.482607  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6db7a8007940d6b8bc088af66589ae" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:00.483287  3431 raft_consensus.cc:3060] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.485324  3553 tablet_bootstrap.cc:654] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.491217  3431 raft_consensus.cc:2468] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:00.492558  3322 leader_election.cc:304] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:00.493479  3623 raft_consensus.cc:2804] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:00.494031  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6db7a8007940d6b8bc088af66589ae" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:00.494231  3623 raft_consensus.cc:697] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:00.493525  3551 ts_tablet_manager.cc:1434] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.001s
I20260709 13:14:00.494675  3283 raft_consensus.cc:3060] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.495252  3623 consensus_queue.cc:237] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.508841  3551 tablet_bootstrap.cc:492] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:00.490178  3552 tablet_bootstrap.cc:654] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.514106  3283 raft_consensus.cc:2468] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:00.518227  3179 catalog_manager.cc:5697] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:00.539093  3551 tablet_bootstrap.cc:654] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.544730  3554 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:00.545382  3554 ts_tablet_manager.cc:1403] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.323s	user 0.011s	sys 0.008s
I20260709 13:14:00.547986  3554 raft_consensus.cc:359] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.548902  3554 raft_consensus.cc:385] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.549247  3554 raft_consensus.cc:740] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.550066  3554 consensus_queue.cc:260] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.552060  3554 ts_tablet_manager.cc:1434] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:00.553273  3554 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:00.568321  3779 raft_consensus.cc:493] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.568902  3779 raft_consensus.cc:515] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.576661  3178 catalog_manager.cc:5697] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:00.577551  3552 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:00.578430  3552 ts_tablet_manager.cc:1403] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.151s	user 0.002s	sys 0.011s
I20260709 13:14:00.582156  3779 leader_election.cc:290] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:00.583382  3552 raft_consensus.cc:359] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.585014  3552 raft_consensus.cc:385] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.585363  3552 raft_consensus.cc:740] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.585611  3554 tablet_bootstrap.cc:654] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.580003  3178 catalog_manager.cc:5697] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:00.574699  3772 raft_consensus.cc:493] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.591884  3772 raft_consensus.cc:515] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.594430  3552 consensus_queue.cc:260] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.594159  3772 leader_election.cc:290] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:00.595506  3506 raft_consensus.cc:3060] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.601166  3782 raft_consensus.cc:493] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.601684  3782 raft_consensus.cc:515] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.603711  3782 leader_election.cc:290] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:00.605203  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:14:00.608994  3396 leader_election.cc:343] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.610072  3355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98065e64beff449b8caef22a3f2a49f5" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:00.612398  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98065e64beff449b8caef22a3f2a49f5" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:00.613734  3552 ts_tablet_manager.cc:1434] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.004s
I20260709 13:14:00.613660  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.620730  3553 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:00.621259  3553 ts_tablet_manager.cc:1403] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.224s	user 0.024s	sys 0.010s
I20260709 13:14:00.621793  3552 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:00.626094  3551 tablet_bootstrap.cc:492] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:00.626607  3551 ts_tablet_manager.cc:1403] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.118s	user 0.017s	sys 0.002s
I20260709 13:14:00.630561  3786 raft_consensus.cc:493] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.632184  3786 raft_consensus.cc:515] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.632566  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.633404  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
W20260709 13:14:00.633659  3324 leader_election.cc:343] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.634069  3428 raft_consensus.cc:2468] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:14:00.634859  3471 leader_election.cc:304] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:00.635918  3554 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:00.636413  3554 ts_tablet_manager.cc:1403] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.083s	user 0.025s	sys 0.011s
I20260709 13:14:00.639165  3554 raft_consensus.cc:359] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.639750  3554 raft_consensus.cc:385] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.639981  3554 raft_consensus.cc:740] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.640640  3554 consensus_queue.cc:260] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.642403  3554 ts_tablet_manager.cc:1434] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:00.644631  3554 tablet_bootstrap.cc:492] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:00.642519  3551 raft_consensus.cc:359] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.646070  3551 raft_consensus.cc:385] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.646363  3551 raft_consensus.cc:740] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.651088  3553 raft_consensus.cc:359] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.651799  3553 raft_consensus.cc:385] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.652079  3553 raft_consensus.cc:740] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.652767  3553 consensus_queue.cc:260] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.647063  3322 leader_election.cc:304] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:00.654065  3779 raft_consensus.cc:2804] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.647708  3774 raft_consensus.cc:2749] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:00.654325  3554 tablet_bootstrap.cc:654] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.654529  3779 raft_consensus.cc:493] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.655004  3779 raft_consensus.cc:3060] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.655848  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.657305  3504 raft_consensus.cc:3060] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:00.659648  3398 leader_election.cc:343] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:00.664266  3249 leader_election.cc:343] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.664703  3398 leader_election.cc:304] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:00.665982  3782 raft_consensus.cc:2749] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:00.669237  3398 consensus_peers.cc:597] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea -> Peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Couldn't send request to peer 73143d6f16c94a3aaef99f8e07588c1f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:00.670255  3551 consensus_queue.cc:260] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.673069  3551 ts_tablet_manager.cc:1434] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.002s
I20260709 13:14:00.673301  3553 ts_tablet_manager.cc:1434] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.001s
I20260709 13:14:00.673965  3551 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:00.674633  3553 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:00.676265  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:00.676920  3428 raft_consensus.cc:2468] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:00.677937  3247 leader_election.cc:304] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:00.678825  3723 raft_consensus.cc:2804] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.679209  3723 raft_consensus.cc:493] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.679513  3723 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.682718  3623 consensus_queue.cc:1048] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.654578  3786 leader_election.cc:290] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
W20260709 13:14:00.687114  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:00.684110  3779 raft_consensus.cc:515] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.696146  3552 tablet_bootstrap.cc:654] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.696753  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:00.697270  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
W20260709 13:14:00.698397  3324 leader_election.cc:343] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.698657  3428 raft_consensus.cc:3060] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.697849  3779 leader_election.cc:290] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:00.689589  3723 raft_consensus.cc:515] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.707448  3723 leader_election.cc:290] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:00.710807  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
W20260709 13:14:00.712383  3249 leader_election.cc:343] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.715437  3553 tablet_bootstrap.cc:654] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.729154  3770 raft_consensus.cc:493] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.729626  3770 raft_consensus.cc:515] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.731537  3770 leader_election.cc:290] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:00.717226  3554 tablet_bootstrap.cc:492] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:00.717012  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:00.732405  3554 ts_tablet_manager.cc:1403] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.089s	user 0.027s	sys 0.017s
I20260709 13:14:00.732754  3432 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.735143  3551 tablet_bootstrap.cc:654] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.735494  3779 consensus_queue.cc:1048] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.738368  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:14:00.740118  3396 leader_election.cc:343] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.735199  3554 raft_consensus.cc:359] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.741331  3554 raft_consensus.cc:385] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.741669  3554 raft_consensus.cc:740] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.742591  3554 consensus_queue.cc:260] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.743882  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.745764  3554 ts_tablet_manager.cc:1434] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 13:14:00.745617  3432 raft_consensus.cc:2468] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:00.748517  3247 leader_election.cc:304] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:00.749336  3723 raft_consensus.cc:2804] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:00.749811  3723 raft_consensus.cc:697] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
W20260709 13:14:00.749881  3398 leader_election.cc:343] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.750366  3398 leader_election.cc:304] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:00.750640  3723 consensus_queue.cc:237] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.751519  3782 raft_consensus.cc:2749] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:00.753360  3179 catalog_manager.cc:5697] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:00.711724  3428 raft_consensus.cc:2468] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:00.757146  3554 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:00.768853  3552 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:00.771020  3322 leader_election.cc:304] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:00.775918  3553 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:00.778995  3623 raft_consensus.cc:2804] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:00.779624  3623 raft_consensus.cc:697] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:00.780659  3623 consensus_queue.cc:237] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.794587  3552 ts_tablet_manager.cc:1403] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.180s	user 0.018s	sys 0.006s
I20260709 13:14:00.805538  3181 catalog_manager.cc:5697] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:00.818281  3551 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:00.820591  3551 ts_tablet_manager.cc:1403] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.147s	user 0.019s	sys 0.026s
I20260709 13:14:00.822594  3552 raft_consensus.cc:359] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.823429  3552 raft_consensus.cc:385] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.823184  3551 raft_consensus.cc:359] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.823725  3552 raft_consensus.cc:740] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.830236  3552 consensus_queue.cc:260] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.832155  3552 ts_tablet_manager.cc:1434] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:14:00.833006  3552 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:00.823905  3551 raft_consensus.cc:385] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.837543  3551 raft_consensus.cc:740] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.838246  3551 consensus_queue.cc:260] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.839730  3552 tablet_bootstrap.cc:654] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.842316  3551 ts_tablet_manager.cc:1434] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:14:00.824363  3553 ts_tablet_manager.cc:1403] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.150s	user 0.021s	sys 0.017s
W20260709 13:14:00.827921  3322 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 57008.5us
I20260709 13:14:00.829622  3554 tablet_bootstrap.cc:654] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.845677  3553 raft_consensus.cc:359] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.846333  3553 raft_consensus.cc:385] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.846612  3553 raft_consensus.cc:740] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.852782  3551 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:00.816572  3653 consensus_queue.cc:1048] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 13:14:00.854482  3623 consensus_queue.cc:1048] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.856117  3554 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:00.856760  3554 ts_tablet_manager.cc:1403] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.000s
I20260709 13:14:00.860319  3554 raft_consensus.cc:359] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.861523  3554 raft_consensus.cc:385] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.861792  3554 raft_consensus.cc:740] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.863230  3554 consensus_queue.cc:260] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.864094  3552 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:00.864576  3552 ts_tablet_manager.cc:1403] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.005s
I20260709 13:14:00.865763  3554 ts_tablet_manager.cc:1434] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:14:00.866739  3554 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:00.867340  3552 raft_consensus.cc:359] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.868024  3552 raft_consensus.cc:385] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.868387  3552 raft_consensus.cc:740] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.869136  3552 consensus_queue.cc:260] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.871479  3623 raft_consensus.cc:493] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.872192  3551 tablet_bootstrap.cc:654] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.871991  3623 raft_consensus.cc:515] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.873385  3554 tablet_bootstrap.cc:654] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.847270  3553 consensus_queue.cc:260] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.875140  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e3b18f25574f46ac56a124bb52c1c3" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:00.876899  3553 ts_tablet_manager.cc:1434] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 13:14:00.877247  3623 leader_election.cc:290] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:00.878446  3553 tablet_bootstrap.cc:492] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:00.878723  3552 ts_tablet_manager.cc:1434] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
W20260709 13:14:00.879508  3324 leader_election.cc:343] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.880017  3552 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:00.880390  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e3b18f25574f46ac56a124bb52c1c3" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:00.881038  3282 raft_consensus.cc:2468] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:14:00.882339  3323 leader_election.cc:304] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:00.884883  3554 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:00.885394  3554 ts_tablet_manager.cc:1403] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 13:14:00.886438  3623 raft_consensus.cc:2804] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.886988  3552 tablet_bootstrap.cc:654] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.887555  3653 raft_consensus.cc:493] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.886911  3623 raft_consensus.cc:493] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.888402  3623 raft_consensus.cc:3060] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.888010  3554 raft_consensus.cc:359] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.888882  3554 raft_consensus.cc:385] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.889170  3554 raft_consensus.cc:740] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.889920  3554 consensus_queue.cc:260] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.892227  3554 ts_tablet_manager.cc:1434] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:00.888116  3653 raft_consensus.cc:515] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.893489  3554 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:00.895463  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304d003e5a044246b2968b09bafc2146" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:00.896162  3282 raft_consensus.cc:2468] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:14:00.898131  3623 raft_consensus.cc:515] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.898451  3397 leader_election.cc:304] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:00.900763  3824 raft_consensus.cc:2804] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.901175  3824 raft_consensus.cc:493] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.900861  3554 tablet_bootstrap.cc:654] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.901468  3623 leader_election.cc:290] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:00.901551  3824 raft_consensus.cc:3060] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.903153  3553 tablet_bootstrap.cc:654] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:00.906767  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e3b18f25574f46ac56a124bb52c1c3" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:00.907655  3504 raft_consensus.cc:3060] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.910853  3552 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:00.911497  3552 ts_tablet_manager.cc:1403] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.000s
I20260709 13:14:00.914783  3772 raft_consensus.cc:493] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.915692  3772 raft_consensus.cc:515] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.914865  3723 raft_consensus.cc:493] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.917969  3723 raft_consensus.cc:515] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.918015  3772 leader_election.cc:290] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:00.919683  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:00.921691  3799 consensus_queue.cc:1048] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.922021  3723 leader_election.cc:290] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:00.922574  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e3b18f25574f46ac56a124bb52c1c3" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:00.923010  3553 tablet_bootstrap.cc:492] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:00.923290  3282 raft_consensus.cc:3060] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.923507  3553 ts_tablet_manager.cc:1403] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.002s
I20260709 13:14:00.926040  3799 consensus_queue.cc:1048] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:00.926234  3553 raft_consensus.cc:359] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.926971  3553 raft_consensus.cc:385] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.927258  3553 raft_consensus.cc:740] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.927899  3553 consensus_queue.cc:260] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:00.927763  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:00.930408  3553 ts_tablet_manager.cc:1434] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:00.930573  3432 raft_consensus.cc:2393] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
I20260709 13:14:00.931387  3553 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:00.932700  3552 raft_consensus.cc:359] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.933481  3552 raft_consensus.cc:385] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.933802  3552 raft_consensus.cc:740] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.935575  3653 leader_election.cc:290] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:00.935603  3470 leader_election.cc:304] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:00.936580  3772 raft_consensus.cc:3060] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.937894  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304d003e5a044246b2968b09bafc2146" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:00.940805  3506 raft_consensus.cc:3060] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.941579  3553 tablet_bootstrap.cc:654] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:00.949123  3396 leader_election.cc:343] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.922163  3504 raft_consensus.cc:2468] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:00.951303  3778 consensus_queue.cc:1048] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:14:00.967265  3324 leader_election.cc:304] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:00.968163  3778 raft_consensus.cc:2804] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:00.968631  3778 raft_consensus.cc:697] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:00.954825  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:00.969733  3778 consensus_queue.cc:237] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.963609  3623 consensus_queue.cc:1048] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:14:00.953634  3772 raft_consensus.cc:2749] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:00.960031  3554 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:00.976786  3554 ts_tablet_manager.cc:1403] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.083s	user 0.017s	sys 0.018s
I20260709 13:14:00.958732  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:00.977384  3356 raft_consensus.cc:2468] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:00.957345  3824 raft_consensus.cc:515] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.960697  3552 consensus_queue.cc:260] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:00.970643  3432 raft_consensus.cc:2468] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:00.979741  3247 leader_election.cc:304] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:00.980538  3733 raft_consensus.cc:2804] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:00.981148  3733 raft_consensus.cc:493] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.981819  3733 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.980496  3554 raft_consensus.cc:359] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:00.982460  3554 raft_consensus.cc:385] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:00.982385  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304d003e5a044246b2968b09bafc2146" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:00.983175  3280 raft_consensus.cc:3060] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.985327  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304d003e5a044246b2968b09bafc2146" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
W20260709 13:14:00.986804  3396 leader_election.cc:343] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:00.989349  3733 raft_consensus.cc:515] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:00.991410  3733 leader_election.cc:290] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:00.981148  3552 ts_tablet_manager.cc:1434] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.003s
I20260709 13:14:00.993008  3552 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:00.992313  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:00.994396  3432 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:00.997864  3554 raft_consensus.cc:740] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:00.999123  3554 consensus_queue.cc:260] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.000161  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:01.001024  3356 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.002526  3554 ts_tablet_manager.cc:1434] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:14:00.891824  3180 catalog_manager.cc:5697] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:01.003783  3552 tablet_bootstrap.cc:654] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.004207  3824 leader_election.cc:290] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:01.009793  3356 raft_consensus.cc:2468] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:01.025334  3247 leader_election.cc:304] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:01.026190  3733 raft_consensus.cc:2804] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:01.026662  3733 raft_consensus.cc:697] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:01.027496  3733 consensus_queue.cc:237] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.010340  3432 raft_consensus.cc:2468] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:01.019701  3623 raft_consensus.cc:493] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.022552  3282 raft_consensus.cc:2468] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:01.035460  3623 raft_consensus.cc:515] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.039448  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596e0222214f40b5861691b7bcaeea24" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:01.040128  3429 raft_consensus.cc:2468] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 0.
I20260709 13:14:01.041358  3322 leader_election.cc:304] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:01.041577  3506 raft_consensus.cc:3060] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.044209  3778 raft_consensus.cc:2804] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:01.044597  3778 raft_consensus.cc:493] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.044955  3778 raft_consensus.cc:3060] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.046351  3554 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:01.047622  3623 leader_election.cc:290] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:01.053125  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596e0222214f40b5861691b7bcaeea24" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:01.057430  3280 raft_consensus.cc:2468] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:01.065867  3397 leader_election.cc:304] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
W20260709 13:14:01.066191  3324 leader_election.cc:343] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.066852  3824 raft_consensus.cc:2804] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:01.067338  3824 raft_consensus.cc:697] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:01.064486  3778 raft_consensus.cc:515] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.069687  3824 consensus_queue.cc:237] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.071260  3778 leader_election.cc:290] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:01.071908  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596e0222214f40b5861691b7bcaeea24" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:01.072561  3432 raft_consensus.cc:3060] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.074190  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596e0222214f40b5861691b7bcaeea24" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:01.075999  3553 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:01.076625  3553 ts_tablet_manager.cc:1403] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.145s	user 0.040s	sys 0.046s
I20260709 13:14:01.081264  3551 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:01.081859  3551 ts_tablet_manager.cc:1403] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.229s	user 0.014s	sys 0.005s
I20260709 13:14:01.083057  3553 raft_consensus.cc:359] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.083982  3553 raft_consensus.cc:385] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.084355  3553 raft_consensus.cc:740] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.089941  3551 raft_consensus.cc:359] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.091193  3551 raft_consensus.cc:385] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:14:01.096233  3324 leader_election.cc:343] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.108486  3551 raft_consensus.cc:740] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.100638  3554 tablet_bootstrap.cc:654] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.109268  3551 consensus_queue.cc:260] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.109673  3178 catalog_manager.cc:5697] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:01.114280  3551 ts_tablet_manager.cc:1434] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:14:01.104535  3553 consensus_queue.cc:260] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.116536  3551 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:01.123301  3553 ts_tablet_manager.cc:1434] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:14:01.124905  3553 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:01.132035  3552 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:01.132603  3552 ts_tablet_manager.cc:1403] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.140s	user 0.019s	sys 0.011s
I20260709 13:14:01.136961  3552 raft_consensus.cc:359] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.137710  3552 raft_consensus.cc:385] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.138075  3552 raft_consensus.cc:740] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.138880  3552 consensus_queue.cc:260] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.140578  3551 tablet_bootstrap.cc:654] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.141229  3552 ts_tablet_manager.cc:1434] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:14:01.142714  3552 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:01.142782  3180 catalog_manager.cc:5697] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:01.146618  3824 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.157292  3824 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.154553  3553 tablet_bootstrap.cc:654] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.168586  3824 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:01.171465  3552 tablet_bootstrap.cc:654] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.172210  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:14:01.173765  3397 leader_election.cc:343] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.175006  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:01.178742  3554 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:01.179237  3554 ts_tablet_manager.cc:1403] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.133s	user 0.013s	sys 0.018s
W20260709 13:14:01.182863  3398 leader_election.cc:343] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.183266  3398 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:01.184437  3181 catalog_manager.cc:5697] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:01.183956  3824 raft_consensus.cc:2749] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:01.185328  3432 raft_consensus.cc:2468] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:01.195089  3322 leader_election.cc:304] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:01.196455  3552 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:01.196822  3623 raft_consensus.cc:2804] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:01.196956  3552 ts_tablet_manager.cc:1403] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.007s
I20260709 13:14:01.197376  3623 raft_consensus.cc:697] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:01.198282  3623 consensus_queue.cc:237] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.200555  3554 raft_consensus.cc:359] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.201215  3554 raft_consensus.cc:385] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.201462  3554 raft_consensus.cc:740] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.202203  3554 consensus_queue.cc:260] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.210537  3553 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:01.211090  3553 ts_tablet_manager.cc:1403] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.086s	user 0.007s	sys 0.009s
I20260709 13:14:01.210947  3552 raft_consensus.cc:359] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.214500  3553 raft_consensus.cc:359] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.215214  3553 raft_consensus.cc:385] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.215517  3553 raft_consensus.cc:740] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.218281  3554 ts_tablet_manager.cc:1434] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 13:14:01.219359  3554 tablet_bootstrap.cc:492] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:01.223656  3506 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.211931  3552 raft_consensus.cc:385] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.225461  3554 tablet_bootstrap.cc:654] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.225402  3552 raft_consensus.cc:740] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.226763  3552 consensus_queue.cc:260] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.228420  3552 ts_tablet_manager.cc:1434] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:14:01.229418  3552 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:01.229599  3723 raft_consensus.cc:493] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.230080  3723 raft_consensus.cc:515] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.233644  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:01.235371  3552 tablet_bootstrap.cc:654] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.235253  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:14:01.239451  3249 leader_election.cc:343] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.240332  3723 leader_election.cc:290] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:01.243903  3554 tablet_bootstrap.cc:492] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:01.244400  3554 ts_tablet_manager.cc:1403] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.002s
I20260709 13:14:01.246978  3554 raft_consensus.cc:359] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.247733  3554 raft_consensus.cc:385] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.247995  3554 raft_consensus.cc:740] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.248891  3554 consensus_queue.cc:260] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.251298  3554 ts_tablet_manager.cc:1434] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:01.242986  3181 catalog_manager.cc:5697] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:01.252295  3554 tablet_bootstrap.cc:492] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:01.255506  3799 consensus_queue.cc:1048] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:01.259122  3551 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:01.259581  3551 ts_tablet_manager.cc:1403] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.145s	user 0.014s	sys 0.015s
I20260709 13:14:01.262233  3551 raft_consensus.cc:359] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.262847  3551 raft_consensus.cc:385] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.263101  3551 raft_consensus.cc:740] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.263339  3733 consensus_queue.cc:1048] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:01.263689  3551 consensus_queue.cc:260] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.265908  3551 ts_tablet_manager.cc:1434] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:14:01.268179  3551 tablet_bootstrap.cc:492] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:01.273840  3551 tablet_bootstrap.cc:654] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.278690  3552 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:01.279173  3552 ts_tablet_manager.cc:1403] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.011s
I20260709 13:14:01.281108  3554 tablet_bootstrap.cc:654] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.281718  3552 raft_consensus.cc:359] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.282217  3552 raft_consensus.cc:385] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.282388  3552 raft_consensus.cc:740] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.282862  3552 consensus_queue.cc:260] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.216565  3553 consensus_queue.cc:260] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.284052  3552 ts_tablet_manager.cc:1434] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:14:01.288221  3778 raft_consensus.cc:493] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.288815  3552 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:01.288654  3778 raft_consensus.cc:515] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.290515  3357 raft_consensus.cc:2468] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:01.294606  3553 ts_tablet_manager.cc:1434] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.083s	user 0.003s	sys 0.003s
I20260709 13:14:01.294876  3554 tablet_bootstrap.cc:492] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:01.295390  3554 ts_tablet_manager.cc:1403] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.000s
I20260709 13:14:01.295444  3553 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:01.298069  3554 raft_consensus.cc:359] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.300755  3554 raft_consensus.cc:385] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.300983  3554 raft_consensus.cc:740] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.311507  3554 consensus_queue.cc:260] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.314250  3554 ts_tablet_manager.cc:1434] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:14:01.312026  3803 consensus_queue.cc:1048] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:14:01.315187  3554 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:01.310285  3247 leader_election.cc:304] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:01.315986  3786 raft_consensus.cc:2804] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:01.316354  3786 raft_consensus.cc:493] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.316687  3786 raft_consensus.cc:3060] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.319057  3552 tablet_bootstrap.cc:654] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.321395  3554 tablet_bootstrap.cc:654] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.323002  3778 leader_election.cc:290] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:01.331952  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:14:01.333581  3324 leader_election.cc:343] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.340557  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:01.342365  3733 consensus_queue.cc:1048] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:01.363395  3786 raft_consensus.cc:515] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.372601  3280 raft_consensus.cc:2393] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:14:01.381212  3323 leader_election.cc:304] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:01.382323  3553 tablet_bootstrap.cc:654] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.382508  3779 raft_consensus.cc:3060] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.385890  3552 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:01.386425  3552 ts_tablet_manager.cc:1403] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.002s
I20260709 13:14:01.389124  3552 raft_consensus.cc:359] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.389849  3552 raft_consensus.cc:385] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.391346  3551 tablet_bootstrap.cc:492] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:01.392468  3551 ts_tablet_manager.cc:1403] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.125s	user 0.013s	sys 0.014s
I20260709 13:14:01.390118  3552 raft_consensus.cc:740] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.396102  3779 raft_consensus.cc:2749] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:01.398542  3554 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:01.399008  3554 ts_tablet_manager.cc:1403] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.084s	user 0.030s	sys 0.031s
I20260709 13:14:01.400799  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:01.400323  3786 leader_election.cc:290] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:01.404354  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:01.401942  3552 consensus_queue.cc:260] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.401640  3554 raft_consensus.cc:359] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.406818  3554 raft_consensus.cc:385] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.407119  3554 raft_consensus.cc:740] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.409598  3860 raft_consensus.cc:493] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.410899  3860 raft_consensus.cc:515] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.438472  3552 ts_tablet_manager.cc:1434] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.002s
I20260709 13:14:01.438361  3554 consensus_queue.cc:260] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.441392  3552 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:01.443955  3553 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:01.444453  3553 ts_tablet_manager.cc:1403] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.149s	user 0.019s	sys 0.025s
I20260709 13:14:01.446941  3551 raft_consensus.cc:359] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.447932  3551 raft_consensus.cc:385] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.448297  3551 raft_consensus.cc:740] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.449316  3551 consensus_queue.cc:260] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.455914  3551 ts_tablet_manager.cc:1434] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.061s	user 0.003s	sys 0.003s
I20260709 13:14:01.467344  3803 consensus_queue.cc:1048] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:01.467895  3357 raft_consensus.cc:2468] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:01.451395  3503 raft_consensus.cc:3060] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.484169  3554 ts_tablet_manager.cc:1434] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.085s	user 0.013s	sys 0.018s
I20260709 13:14:01.485424  3554 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:01.488502  3552 tablet_bootstrap.cc:654] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.492955  3551 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:01.493315  3553 raft_consensus.cc:359] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.494234  3553 raft_consensus.cc:385] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.494529  3553 raft_consensus.cc:740] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.494731  3860 leader_election.cc:290] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:01.495214  3553 consensus_queue.cc:260] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.546530  3247 leader_election.cc:304] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:01.578886  3620 raft_consensus.cc:2804] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:01.585229  3667 raft_consensus.cc:493] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.586083  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:01.591622  3623 consensus_queue.cc:1048] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:01.595216  3653 raft_consensus.cc:493] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.602524  3429 raft_consensus.cc:2468] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:01.602864  3653 raft_consensus.cc:515] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.605792  3653 leader_election.cc:290] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:01.605508  3723 raft_consensus.cc:493] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.598255  3551 tablet_bootstrap.cc:654] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.611918  3860 raft_consensus.cc:493] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.606912  3723 raft_consensus.cc:515] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.642443  3860 raft_consensus.cc:515] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.643754  3868 raft_consensus.cc:493] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.644469  3855 consensus_queue.cc:1048] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:01.644555  3868 raft_consensus.cc:515] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.647388  3868 leader_election.cc:290] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:01.648741  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e514dae1f1814a2b80b772b07399cc58" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:01.650216  3283 raft_consensus.cc:2468] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:14:01.651921  3667 raft_consensus.cc:515] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.658360  3554 tablet_bootstrap.cc:654] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.661092  3782 raft_consensus.cc:493] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.661827  3782 raft_consensus.cc:515] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:01.662365  3470 leader_election.cc:304] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:01.666694  3861 raft_consensus.cc:2804] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:01.667181  3861 raft_consensus.cc:493] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.667575  3861 raft_consensus.cc:3060] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.667950  3553 ts_tablet_manager.cc:1434] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.223s	user 0.001s	sys 0.006s
I20260709 13:14:01.669866  3553 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:01.671502  3782 leader_election.cc:290] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:01.671835  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:01.675077  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e514dae1f1814a2b80b772b07399cc58" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:01.675985  3786 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.676862  3786 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.678334  3552 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:01.682902  3860 leader_election.cc:290] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:01.683626  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:01.684645  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85719491f2464bbf9c92b2f19238beeb" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:01.683113  3357 raft_consensus.cc:3060] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.655678  3667 leader_election.cc:290] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:01.707654  3860 raft_consensus.cc:493] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.706065  3356 raft_consensus.cc:2468] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:14:01.714360  3552 ts_tablet_manager.cc:1403] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.273s	user 0.004s	sys 0.019s
I20260709 13:14:01.716190  3770 consensus_queue.cc:1048] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:14:01.718824  3733 raft_consensus.cc:493] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.719574  3733 raft_consensus.cc:515] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.720223  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85719491f2464bbf9c92b2f19238beeb" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:01.722239  3733 leader_election.cc:290] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:01.724848  3860 raft_consensus.cc:515] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.727725  3860 leader_election.cc:290] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:01.728513  3723 leader_election.cc:290] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:01.709367  3620 raft_consensus.cc:697] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:01.738911  3786 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:01.740413  3620 consensus_queue.cc:237] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.743386  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:01.744136  3282 raft_consensus.cc:2468] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:01.740130  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304d003e5a044246b2968b09bafc2146" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:01.749171  3503 raft_consensus.cc:2468] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
W20260709 13:14:01.703174  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 93717.6us
I20260709 13:14:01.750910  3772 raft_consensus.cc:493] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.753149  3772 raft_consensus.cc:515] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.754071  3471 leader_election.cc:304] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:01.755293  3862 raft_consensus.cc:2804] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:01.755712  3862 raft_consensus.cc:493] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.756171  3862 raft_consensus.cc:3060] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:01.733866  3324 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 116315us
I20260709 13:14:01.767695  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:01.756570  3397 leader_election.cc:304] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
W20260709 13:14:01.772199  3397 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 67902.4us
I20260709 13:14:01.772310  3770 raft_consensus.cc:2804] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:01.767849  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:01.772790  3770 raft_consensus.cc:493] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.773258  3770 raft_consensus.cc:3060] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.773687  3772 leader_election.cc:290] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
W20260709 13:14:01.776326  3396 leader_election.cc:343] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:01.762148  3398 leader_election.cc:343] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:01.777446  3398 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 44137.2us
W20260709 13:14:01.782007  3396 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 76649.2us
W20260709 13:14:01.777462  3397 leader_election.cc:343] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.791796  3502 raft_consensus.cc:3060] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.796468  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:01.799260  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:01.800212  3428 raft_consensus.cc:2468] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:01.800653  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:01.801585  3431 raft_consensus.cc:2468] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
W20260709 13:14:01.803189  3396 leader_election.cc:343] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.803691  3396 leader_election.cc:304] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:01.796778  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16129126724cb3a6e2ecfa3547b4e6" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:01.809271  3653 raft_consensus.cc:2749] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:01.810699  3396 leader_election.cc:304] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
W20260709 13:14:01.812582  3867 log.cc:927] Time spent T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:14:01.812243  3552 raft_consensus.cc:359] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.813136  3552 raft_consensus.cc:385] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.813514  3552 raft_consensus.cc:740] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.814054  3552 consensus_queue.cc:260] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.815693  3552 ts_tablet_manager.cc:1434] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.085s	user 0.004s	sys 0.000s
W20260709 13:14:01.814350  3249 leader_election.cc:343] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.820956  3782 raft_consensus.cc:2804] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:01.825788  3782 raft_consensus.cc:493] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.826190  3782 raft_consensus.cc:3060] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.841333  3553 tablet_bootstrap.cc:654] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.848173  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596e0222214f40b5861691b7bcaeea24" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:01.849507  3431 raft_consensus.cc:2393] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
I20260709 13:14:01.857913  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:01.858623  3431 raft_consensus.cc:2468] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:01.859148  3552 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
W20260709 13:14:01.863793  3249 leader_election.cc:343] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:01.867107  3247 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:01.873942  3770 raft_consensus.cc:515] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:14:01.877519  3323 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 300062us
I20260709 13:14:01.880246  3551 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:01.880939  3551 ts_tablet_manager.cc:1403] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.388s	user 0.022s	sys 0.004s
I20260709 13:14:01.895809  3554 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:01.897878  3855 raft_consensus.cc:2804] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:01.898379  3855 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.898741  3855 raft_consensus.cc:3060] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.907617  3862 raft_consensus.cc:515] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.909204  3505 raft_consensus.cc:2393] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 972a4469a4654416ab0a153d8c740282 in current term 1: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:01.910424  3862 leader_election.cc:290] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:01.911528  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:01.912155  3282 raft_consensus.cc:3060] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.913398  3554 ts_tablet_manager.cc:1403] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.428s	user 0.018s	sys 0.017s
I20260709 13:14:01.921180  3554 raft_consensus.cc:359] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.922142  3554 raft_consensus.cc:385] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.922510  3554 raft_consensus.cc:740] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.920873  3282 raft_consensus.cc:2468] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:01.923337  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:01.923305  3554 consensus_queue.cc:260] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:01.924160  3432 raft_consensus.cc:3060] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.924252  3471 leader_election.cc:304] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:01.925251  3860 raft_consensus.cc:2804] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:01.925884  3860 raft_consensus.cc:697] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:01.927067  3860 consensus_queue.cc:237] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.930773  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:01.934137  3554 ts_tablet_manager.cc:1434] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:14:01.934759  3551 raft_consensus.cc:359] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.935688  3551 raft_consensus.cc:385] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:01.935871  3554 tablet_bootstrap.cc:492] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:01.936021  3551 raft_consensus.cc:740] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:01.940542  3770 leader_election.cc:290] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:01.943915  3870 consensus_queue.cc:1048] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:01.946192  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16129126724cb3a6e2ecfa3547b4e6" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:01.946625  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304d003e5a044246b2968b09bafc2146" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:01.950004  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e514dae1f1814a2b80b772b07399cc58" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:01.950316  3554 tablet_bootstrap.cc:654] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:01.950623  3280 raft_consensus.cc:3060] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:01.900097  3861 raft_consensus.cc:515] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
W20260709 13:14:01.956945  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 89986.1us
I20260709 13:14:01.957820  3247 leader_election.cc:304] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:01.958508  3861 leader_election.cc:290] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:01.958679  3620 raft_consensus.cc:2764] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:14:01.964725  3432 raft_consensus.cc:2468] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:01.937052  3551 consensus_queue.cc:260] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.988790  3551 ts_tablet_manager.cc:1434] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.108s	user 0.008s	sys 0.006s
I20260709 13:14:01.993510  3872 raft_consensus.cc:493] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.994194  3872 raft_consensus.cc:515] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:01.994352  3855 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:01.986158  3861 raft_consensus.cc:493] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.999141  3861 raft_consensus.cc:515] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.006639  3872 leader_election.cc:290] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:02.014717  3770 consensus_queue.cc:1048] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:02.017546  3552 tablet_bootstrap.cc:654] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.018698  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e514dae1f1814a2b80b772b07399cc58" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:02.028232  3855 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:02.039919  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596e0222214f40b5861691b7bcaeea24" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:02.045222  3280 raft_consensus.cc:2468] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:02.029181  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:02.079027  3551 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:02.071508  3861 leader_election.cc:290] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
W20260709 13:14:02.084671  3880 log.cc:927] Time spent T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Append to log took a long time: real 0.106s	user 0.001s	sys 0.004s
I20260709 13:14:02.085407  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:02.086124  3427 raft_consensus.cc:3060] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.086490  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:02.091871  3620 consensus_queue.cc:1048] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:14:02.094936  3396 leader_election.cc:343] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.061955  3429 raft_consensus.cc:3060] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.086884  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:02.114724  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16129126724cb3a6e2ecfa3547b4e6" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:02.116214  3279 raft_consensus.cc:2468] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:02.118613  3778 consensus_queue.cc:1048] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:02.120476  3397 leader_election.cc:304] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:02.120950  3323 leader_election.cc:304] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:02.114969  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:02.122205  3356 raft_consensus.cc:2468] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:02.125572  3471 leader_election.cc:304] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
W20260709 13:14:02.127296  3470 leader_election.cc:343] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.128545  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:02.122231  3553 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:02.129746  3553 ts_tablet_manager.cc:1403] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.460s	user 0.027s	sys 0.024s
I20260709 13:14:02.131278  3554 tablet_bootstrap.cc:492] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:02.131760  3554 ts_tablet_manager.cc:1403] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.196s	user 0.011s	sys 0.006s
I20260709 13:14:02.132951  3553 raft_consensus.cc:359] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.134249  3824 raft_consensus.cc:2804] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:02.134071  3782 raft_consensus.cc:515] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.134809  3824 raft_consensus.cc:697] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:02.135778  3824 consensus_queue.cc:237] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.137758  3782 leader_election.cc:290] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:02.133729  3553 raft_consensus.cc:385] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.139878  3553 raft_consensus.cc:740] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.139482  3426 raft_consensus.cc:2393] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 972a4469a4654416ab0a153d8c740282 in current term 1: Already voted for candidate 06964934536148c2aa77a2f1c6382fea in this term.
I20260709 13:14:02.110252  3180 catalog_manager.cc:5697] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:02.111198  3247 leader_election.cc:343] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.154789  3355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:02.156174  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:02.157323  3552 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:02.162967  3552 ts_tablet_manager.cc:1403] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.304s	user 0.015s	sys 0.012s
I20260709 13:14:02.148367  3553 consensus_queue.cc:260] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.152637  3861 raft_consensus.cc:2804] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:02.168370  3355 raft_consensus.cc:2468] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:02.167150  3356 raft_consensus.cc:3060] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.177635  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:02.178296  3279 raft_consensus.cc:3060] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.179474  3427 raft_consensus.cc:2468] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:02.169747  3554 raft_consensus.cc:359] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.163754  3247 leader_election.cc:304] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:02.183059  3786 raft_consensus.cc:2749] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:02.183296  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 76365.8us
I20260709 13:14:02.168785  3861 raft_consensus.cc:493] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.185673  3861 raft_consensus.cc:3060] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.191658  3470 leader_election.cc:304] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2
W20260709 13:14:02.192791  3470 leader_election.cc:343] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.181373  3554 raft_consensus.cc:385] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.195585  3553 ts_tablet_manager.cc:1434] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.003s
I20260709 13:14:02.195812  3551 tablet_bootstrap.cc:654] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.194159  3554 raft_consensus.cc:740] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.199494  3667 raft_consensus.cc:2749] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:02.200181  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:02.208144  3554 consensus_queue.cc:260] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.205359  3860 raft_consensus.cc:2749] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:02.217535  3552 raft_consensus.cc:359] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.195071  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16129126724cb3a6e2ecfa3547b4e6" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:02.218829  3552 raft_consensus.cc:385] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.218772  3247 leader_election.cc:304] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:02.219736  3552 raft_consensus.cc:740] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
W20260709 13:14:02.221021  3249 leader_election.cc:343] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.220695  3552 consensus_queue.cc:260] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.225968  3279 raft_consensus.cc:2468] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:02.227770  3552 ts_tablet_manager.cc:1434] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260709 13:14:02.219612  3353 raft_consensus.cc:2468] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
W20260709 13:14:02.229140  3882 log.cc:927] Time spent T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Append to log took a long time: real 0.249s	user 0.006s	sys 0.000s
I20260709 13:14:02.229542  3553 tablet_bootstrap.cc:492] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:02.229746  3855 raft_consensus.cc:2764] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:14:02.235816  3356 raft_consensus.cc:2468] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:02.236860  3429 raft_consensus.cc:2468] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:02.237509  3396 leader_election.cc:304] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:02.238603  3770 raft_consensus.cc:2804] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:02.239148  3770 raft_consensus.cc:697] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:02.240127  3770 consensus_queue.cc:237] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.236459  3552 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:02.250926  3470 leader_election.cc:304] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:02.251716  3860 raft_consensus.cc:2804] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:02.252378  3860 raft_consensus.cc:697] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:02.254038  3554 ts_tablet_manager.cc:1434] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.122s	user 0.009s	sys 0.000s
I20260709 13:14:02.253614  3860 consensus_queue.cc:237] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.254438  3552 tablet_bootstrap.cc:654] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.255221  3554 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:02.252655  3861 raft_consensus.cc:515] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.266130  3247 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:02.267185  3855 raft_consensus.cc:2804] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:02.267748  3855 raft_consensus.cc:697] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:02.268688  3855 consensus_queue.cc:237] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.282483  3553 tablet_bootstrap.cc:654] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.300504  3179 catalog_manager.cc:5697] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:02.303121  3874 consensus_queue.cc:1048] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:02.300144  3470 leader_election.cc:304] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:02.304978  3865 raft_consensus.cc:2804] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:02.305365  3865 raft_consensus.cc:493] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.305704  3865 raft_consensus.cc:3060] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:02.315083  3865 raft_consensus.cc:515] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.324882  3551 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:02.325457  3551 ts_tablet_manager.cc:1403] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.247s	user 0.017s	sys 0.004s
I20260709 13:14:02.333070  3506 raft_consensus.cc:1240] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Rejecting Update request from peer 972a4469a4654416ab0a153d8c740282 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:14:02.336036  3872 consensus_queue.cc:1048] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:02.337649  3895 raft_consensus.cc:493] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.338192  3895 raft_consensus.cc:515] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.342444  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:14:02.344313  3398 leader_election.cc:343] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.345108  3862 consensus_queue.cc:1048] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:02.346813  3861 leader_election.cc:290] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:02.347904  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16129126724cb3a6e2ecfa3547b4e6" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:02.348541  3282 raft_consensus.cc:3060] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.349133  3551 raft_consensus.cc:359] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.350015  3551 raft_consensus.cc:385] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.350329  3551 raft_consensus.cc:740] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.352075  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16129126724cb3a6e2ecfa3547b4e6" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:02.351114  3551 consensus_queue.cc:260] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.357126  3353 raft_consensus.cc:3060] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.354940  3895 leader_election.cc:290] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:02.381309  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:02.382093  3279 raft_consensus.cc:2468] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:14:02.385866  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:02.386286  3355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:02.385378  3733 consensus_queue.cc:1059] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:02.388192  3733 raft_consensus.cc:3055] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:02.388665  3733 raft_consensus.cc:740] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:02.388984  3470 leader_election.cc:304] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:02.389469  3733 consensus_queue.cc:260] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.409626  3894 raft_consensus.cc:493] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.390149  3861 raft_consensus.cc:2749] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:14:02.410308  3894 raft_consensus.cc:515] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.391515  3865 leader_election.cc:290] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:02.409742  3397 leader_election.cc:304] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:02.409715  3733 raft_consensus.cc:3060] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:02.414856  3782 raft_consensus.cc:2804] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:02.415067  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:02.415370  3782 raft_consensus.cc:493] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.415817  3782 raft_consensus.cc:3060] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.416865  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:02.412969  3894 leader_election.cc:290] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:02.419122  3551 ts_tablet_manager.cc:1434] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.003s
W20260709 13:14:02.419808  3398 leader_election.cc:343] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.421597  3554 tablet_bootstrap.cc:654] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.422999  3782 raft_consensus.cc:515] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.420477  3551 tablet_bootstrap.cc:492] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:02.430387  3862 raft_consensus.cc:493] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.431809  3553 tablet_bootstrap.cc:492] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:02.433187  3553 ts_tablet_manager.cc:1403] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.204s	user 0.021s	sys 0.002s
I20260709 13:14:02.430903  3862 raft_consensus.cc:515] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.439637  3862 leader_election.cc:290] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:02.429656  3282 raft_consensus.cc:2468] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:02.441252  3355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:02.441644  3471 leader_election.cc:304] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:02.442724  3862 raft_consensus.cc:2804] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:02.443349  3862 raft_consensus.cc:697] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
W20260709 13:14:02.444262  3470 leader_election.cc:343] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.447868  3551 tablet_bootstrap.cc:654] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:02.437897  3397 leader_election.cc:343] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.453472  3397 leader_election.cc:304] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:02.454813  3553 raft_consensus.cc:359] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.455255  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:02.456439  3429 raft_consensus.cc:2468] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:02.445762  3862 consensus_queue.cc:237] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.458258  3470 leader_election.cc:304] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:02.459573  3865 raft_consensus.cc:2804] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:02.460003  3865 raft_consensus.cc:493] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.460549  3865 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.455646  3770 raft_consensus.cc:2749] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:02.464238  3861 consensus_queue.cc:1048] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:02.470320  3865 raft_consensus.cc:515] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.471336  3553 raft_consensus.cc:385] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.471869  3553 raft_consensus.cc:740] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.472738  3553 consensus_queue.cc:260] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.477650  3552 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:02.478171  3552 ts_tablet_manager.cc:1403] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.249s	user 0.015s	sys 0.014s
I20260709 13:14:02.438468  3353 raft_consensus.cc:2468] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:02.488834  3553 ts_tablet_manager.cc:1434] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.055s	user 0.004s	sys 0.003s
I20260709 13:14:02.492718  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:02.493695  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:02.494319  3282 raft_consensus.cc:3060] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.494549  3865 leader_election.cc:290] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
W20260709 13:14:02.496858  3398 leader_election.cc:343] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.496261  3782 leader_election.cc:290] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:02.498102  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:02.502996  3551 tablet_bootstrap.cc:492] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:02.503810  3551 ts_tablet_manager.cc:1403] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.084s	user 0.008s	sys 0.021s
I20260709 13:14:02.504148  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:02.504786  3553 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:02.501523  3428 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.517418  3551 raft_consensus.cc:359] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.518443  3551 raft_consensus.cc:385] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.518844  3551 raft_consensus.cc:740] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.519912  3553 tablet_bootstrap.cc:654] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.519690  3551 consensus_queue.cc:260] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
W20260709 13:14:02.523340  3470 leader_election.cc:343] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.524150  3869 raft_consensus.cc:493] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.524677  3869 raft_consensus.cc:515] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.524598  3552 raft_consensus.cc:359] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.528182  3552 raft_consensus.cc:385] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.528533  3552 raft_consensus.cc:740] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.530028  3554 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:02.529945  3552 consensus_queue.cc:260] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.530017  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:02.530745  3554 ts_tablet_manager.cc:1403] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.276s	user 0.020s	sys 0.003s
I20260709 13:14:02.531775  3430 raft_consensus.cc:2393] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:02.532349  3869 leader_election.cc:290] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:02.533977  3551 ts_tablet_manager.cc:1434] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 13:14:02.533592  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:02.536369  3324 leader_election.cc:304] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:02.537891  3667 raft_consensus.cc:3060] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.542841  3282 raft_consensus.cc:2468] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:02.543996  3397 leader_election.cc:304] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:02.544729  3551 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:02.545167  3782 raft_consensus.cc:2804] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:02.545938  3782 raft_consensus.cc:697] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:02.547063  3552 ts_tablet_manager.cc:1434] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 13:14:02.546905  3554 raft_consensus.cc:359] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.551270  3554 raft_consensus.cc:385] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.551580  3554 raft_consensus.cc:740] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.552314  3782 consensus_queue.cc:237] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.553192  3667 raft_consensus.cc:2749] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:02.554628  3552 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:02.555855  3554 consensus_queue.cc:260] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.558990  3553 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:02.559715  3553 ts_tablet_manager.cc:1403] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.059s	user 0.009s	sys 0.005s
I20260709 13:14:02.562789  3551 tablet_bootstrap.cc:654] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.574231  3181 catalog_manager.cc:5697] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:02.556355  3428 raft_consensus.cc:2468] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:02.621483  3470 leader_election.cc:304] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:02.624262  3865 raft_consensus.cc:2804] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:02.627151  3865 raft_consensus.cc:697] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:02.627873  3553 raft_consensus.cc:359] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.628607  3553 raft_consensus.cc:385] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.628851  3553 raft_consensus.cc:740] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.633193  3554 ts_tablet_manager.cc:1434] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.102s	user 0.011s	sys 0.000s
I20260709 13:14:02.636408  3865 consensus_queue.cc:237] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.642268  3894 consensus_queue.cc:1048] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:02.651724  3552 tablet_bootstrap.cc:654] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.665776  3653 consensus_queue.cc:1048] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:02.675669  3553 consensus_queue.cc:260] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.676589  3733 consensus_queue.cc:1048] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:02.679005  3553 ts_tablet_manager.cc:1434] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.119s	user 0.007s	sys 0.000s
I20260709 13:14:02.680076  3553 tablet_bootstrap.cc:492] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:02.682025  3180 catalog_manager.cc:5697] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:02.681306  3862 consensus_queue.cc:1048] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:14:02.715279  3554 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:02.734817  3503 raft_consensus.cc:3060] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.739619  3350 raft_consensus.cc:3060] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:02.764569  3865 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.765501  3865 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:02.767302  3895 raft_consensus.cc:493] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.767925  3895 raft_consensus.cc:515] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.784704  3733 consensus_queue.cc:1048] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:02.813268  3551 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:02.813823  3551 ts_tablet_manager.cc:1403] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.269s	user 0.032s	sys 0.053s
I20260709 13:14:02.785046  3178 catalog_manager.cc:5697] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } }
I20260709 13:14:02.822459  3895 leader_election.cc:290] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:02.823571  3865 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
W20260709 13:14:02.830698  3920 log.cc:927] Time spent T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:14:02.826160  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63dbef5a847a4451b6e4e46d15341334" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:02.833118  3282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:02.834282  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63dbef5a847a4451b6e4e46d15341334" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:14:02.837703  3396 leader_election.cc:343] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.845760  3552 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
W20260709 13:14:02.846829  3398 leader_election.cc:343] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:02.847329  3398 leader_election.cc:304] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:02.848163  3553 tablet_bootstrap.cc:654] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.848785  3782 raft_consensus.cc:2749] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:02.846339  3552 ts_tablet_manager.cc:1403] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.292s	user 0.017s	sys 0.032s
I20260709 13:14:02.877866  3554 tablet_bootstrap.cc:654] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.880430  3782 consensus_queue.cc:1048] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:02.888636  3552 raft_consensus.cc:359] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.866443  3180 catalog_manager.cc:5697] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:02.893091  3552 raft_consensus.cc:385] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.893468  3552 raft_consensus.cc:740] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.897384  3552 consensus_queue.cc:260] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:02.920118  3553 tablet_bootstrap.cc:492] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:02.921597  3553 ts_tablet_manager.cc:1403] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.242s	user 0.019s	sys 0.000s
I20260709 13:14:02.924973  3552 ts_tablet_manager.cc:1434] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.060s	user 0.004s	sys 0.003s
I20260709 13:14:02.931602  3551 raft_consensus.cc:359] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.932369  3551 raft_consensus.cc:385] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:02.932652  3551 raft_consensus.cc:740] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:02.933318  3551 consensus_queue.cc:260] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.934582  3782 consensus_queue.cc:1048] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:02.936455  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:02.936909  3551 ts_tablet_manager.cc:1434] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.123s	user 0.007s	sys 0.000s
I20260709 13:14:02.938100  3551 tablet_bootstrap.cc:492] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:02.945313  3181 catalog_manager.cc:5697] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:02.948071  3181 catalog_manager.cc:5697] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:02.955250  3181 catalog_manager.cc:5697] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:02.956203  3862 consensus_queue.cc:1048] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:02.958688  3895 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.987012  3895 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.959641  3552 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:02.965045  3855 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.989698  3855 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:02.968768  3865 consensus_queue.cc:1048] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:02.992334  3551 tablet_bootstrap.cc:654] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:02.991880  3855 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:02.972932  3553 raft_consensus.cc:359] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:02.999655  3553 raft_consensus.cc:385] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.000113  3553 raft_consensus.cc:740] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.001017  3553 consensus_queue.cc:260] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
W20260709 13:14:03.019167  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 55421.2us
I20260709 13:14:03.020982  3895 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.021737  3553 ts_tablet_manager.cc:1434] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.099s	user 0.008s	sys 0.000s
I20260709 13:14:03.024497  3470 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
W20260709 13:14:03.024722  3398 consensus_peers.cc:597] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea -> Peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Couldn't send request to peer 73143d6f16c94a3aaef99f8e07588c1f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:03.027998  3862 raft_consensus.cc:2749] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:03.031273  3552 tablet_bootstrap.cc:654] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.035210  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:03.046273  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:03.047053  3279 raft_consensus.cc:2468] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:14:03.047900  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:14:03.051802  3396 leader_election.cc:343] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.053481  3554 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.053965  3554 ts_tablet_manager.cc:1403] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.353s	user 0.031s	sys 0.028s
I20260709 13:14:03.056313  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:03.056993  3430 raft_consensus.cc:2468] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:03.059834  3554 raft_consensus.cc:359] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.060669  3554 raft_consensus.cc:385] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.061008  3554 raft_consensus.cc:740] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.060281  3553 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:03.068773  3865 consensus_queue.cc:1048] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:14:03.061695  3554 consensus_queue.cc:260] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.082790  3554 ts_tablet_manager.cc:1434] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.004s
I20260709 13:14:03.083799  3554 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:03.088451  3397 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:03.089305  3895 raft_consensus.cc:2804] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.089656  3895 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.089962  3895 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.092401  3553 tablet_bootstrap.cc:654] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:03.069337  3247 leader_election.cc:343] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.102268  3551 tablet_bootstrap.cc:492] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:03.102809  3551 ts_tablet_manager.cc:1403] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.165s	user 0.034s	sys 0.015s
I20260709 13:14:03.104027  3862 consensus_queue.cc:1048] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:03.105509  3551 raft_consensus.cc:359] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.106220  3551 raft_consensus.cc:385] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.106473  3551 raft_consensus.cc:740] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.107244  3551 consensus_queue.cc:260] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.109447  3247 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:03.122692  3551 ts_tablet_manager.cc:1434] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:14:03.124739  3551 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:03.122702  3931 raft_consensus.cc:2804] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.140635  3931 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.140973  3931 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.145479  3895 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.170246  3782 consensus_queue.cc:1048] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:03.175045  3551 tablet_bootstrap.cc:654] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.185670  3554 tablet_bootstrap.cc:654] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.218977  3352 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.215466  3931 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.238714  3552 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:03.239233  3552 ts_tablet_manager.cc:1403] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.280s	user 0.015s	sys 0.023s
I20260709 13:14:03.241865  3552 raft_consensus.cc:359] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.242273  3855 raft_consensus.cc:493] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.242801  3855 raft_consensus.cc:515] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.243187  3552 raft_consensus.cc:385] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.243677  3552 raft_consensus.cc:740] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.244889  3855 leader_election.cc:290] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.244494  3552 consensus_queue.cc:260] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.228932  3895 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.248689  3931 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:03.231887  3862 raft_consensus.cc:493] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.250492  3895 raft_consensus.cc:493] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.260242  3552 ts_tablet_manager.cc:1434] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:14:03.264854  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:03.264752  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:03.273108  3280 raft_consensus.cc:2393] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 06964934536148c2aa77a2f1c6382fea in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:14:03.273476  3554 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.273978  3554 ts_tablet_manager.cc:1403] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.190s	user 0.007s	sys 0.016s
I20260709 13:14:03.275027  3431 raft_consensus.cc:2393] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 972a4469a4654416ab0a153d8c740282 in current term 1: Already voted for candidate 06964934536148c2aa77a2f1c6382fea in this term.
I20260709 13:14:03.262941  3895 raft_consensus.cc:515] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
W20260709 13:14:03.283885  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:03.284449  3895 leader_election.cc:290] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:03.284775  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:03.285703  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:03.285712  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:03.286708  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:14:03.287547  3398 leader_election.cc:343] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.293257  3553 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:03.293814  3553 ts_tablet_manager.cc:1403] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.252s	user 0.028s	sys 0.001s
W20260709 13:14:03.294801  3396 leader_election.cc:343] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.301576  3396 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
W20260709 13:14:03.302775  3396 leader_election.cc:343] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.303251  3396 leader_election.cc:304] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:03.303782  3553 raft_consensus.cc:359] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.304179  3894 raft_consensus.cc:2749] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:03.304507  3553 raft_consensus.cc:385] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.255064  3862 raft_consensus.cc:515] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.304797  3553 raft_consensus.cc:740] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.304874  3894 raft_consensus.cc:2749] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:03.305512  3553 consensus_queue.cc:260] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.309237  3551 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:03.309736  3551 ts_tablet_manager.cc:1403] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.185s	user 0.021s	sys 0.021s
I20260709 13:14:03.311208  3503 raft_consensus.cc:2468] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:03.312037  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:03.312738  3281 raft_consensus.cc:2468] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:03.314033  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:03.314159  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:03.314679  3353 raft_consensus.cc:2468] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:03.315634  3471 leader_election.cc:304] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
W20260709 13:14:03.315920  3247 leader_election.cc:343] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.299496  3860 consensus_queue.cc:1048] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:14:03.316493  3247 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972a4469a4654416ab0a153d8c740282; no voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:03.317135  3861 raft_consensus.cc:2804] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.317534  3861 raft_consensus.cc:493] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.317749  3667 raft_consensus.cc:493] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.317994  3861 raft_consensus.cc:3060] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.318365  3667 raft_consensus.cc:515] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.295883  3552 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:03.321987  3554 raft_consensus.cc:359] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.322913  3554 raft_consensus.cc:385] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.323273  3554 raft_consensus.cc:740] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.324162  3554 consensus_queue.cc:260] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.331799  3860 raft_consensus.cc:493] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.332317  3860 raft_consensus.cc:515] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.333803  3247 leader_election.cc:304] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:03.334853  3874 raft_consensus.cc:2749] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:03.335459  3874 raft_consensus.cc:2804] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.335778  3874 raft_consensus.cc:493] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.336119  3874 raft_consensus.cc:3060] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.340910  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:03.341641  3352 raft_consensus.cc:2468] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:03.342716  3860 leader_election.cc:290] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:03.343815  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:03.311278  3862 leader_election.cc:290] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.362696  3553 ts_tablet_manager.cc:1434] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.001s
I20260709 13:14:03.364215  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:03.365650  3470 leader_election.cc:304] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
W20260709 13:14:03.355139  3396 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 52395.1us
I20260709 13:14:03.369694  3862 raft_consensus.cc:2749] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:03.370009  3874 raft_consensus.cc:515] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.372474  3551 raft_consensus.cc:359] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.373212  3551 raft_consensus.cc:385] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.374048  3553 tablet_bootstrap.cc:492] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:03.372788  3667 leader_election.cc:290] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:03.380471  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:03.382947  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:03.385262  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:03.374392  3551 raft_consensus.cc:740] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.387748  3554 ts_tablet_manager.cc:1434] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.113s	user 0.009s	sys 0.001s
I20260709 13:14:03.386521  3551 consensus_queue.cc:260] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.391166  3554 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:03.395366  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:03.396051  3356 raft_consensus.cc:3060] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.410773  3322 leader_election.cc:304] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:03.390815  3874 leader_election.cc:290] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.417879  3874 raft_consensus.cc:493] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.418354  3874 raft_consensus.cc:515] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.420966  3874 leader_election.cc:290] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:03.421211  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:03.422041  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:03.422711  3428 raft_consensus.cc:2468] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
W20260709 13:14:03.423791  3249 leader_election.cc:343] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.427073  3247 leader_election.cc:304] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:03.427328  3356 raft_consensus.cc:2468] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:03.427913  3874 raft_consensus.cc:2804] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.428601  3551 ts_tablet_manager.cc:1434] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.119s	user 0.005s	sys 0.003s
I20260709 13:14:03.428743  3874 raft_consensus.cc:493] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.429226  3874 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.432785  3945 raft_consensus.cc:2749] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:03.437057  3874 raft_consensus.cc:515] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.441264  3552 tablet_bootstrap.cc:654] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.442869  3874 leader_election.cc:290] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:03.443367  3247 leader_election.cc:304] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:03.444878  3874 raft_consensus.cc:2804] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:03.446017  3551 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:03.446343  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:03.447082  3874 raft_consensus.cc:697] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:03.447041  3428 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.448132  3874 consensus_queue.cc:237] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.448856  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:03.452272  3861 raft_consensus.cc:515] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.462981  3503 raft_consensus.cc:2393] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 972a4469a4654416ab0a153d8c740282 in current term 1: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:03.453635  3553 tablet_bootstrap.cc:654] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:03.458611  3249 leader_election.cc:343] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.466373  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:03.467933  3861 leader_election.cc:290] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.468784  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:03.469610  3353 raft_consensus.cc:2393] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:14:03.470494  3470 leader_election.cc:304] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:03.471292  3861 raft_consensus.cc:2749] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:03.472594  3551 tablet_bootstrap.cc:654] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.472702  3428 raft_consensus.cc:2468] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:03.474427  3247 leader_election.cc:304] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:03.475440  3931 raft_consensus.cc:2804] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:03.471761  3552 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:03.475921  3931 raft_consensus.cc:697] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:03.476263  3552 ts_tablet_manager.cc:1403] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.181s	user 0.011s	sys 0.001s
I20260709 13:14:03.478353  3931 consensus_queue.cc:237] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.478816  3552 raft_consensus.cc:359] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.479427  3552 raft_consensus.cc:385] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.479668  3552 raft_consensus.cc:740] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.462188  3554 tablet_bootstrap.cc:654] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.490896  3554 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.491484  3554 ts_tablet_manager.cc:1403] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.103s	user 0.011s	sys 0.000s
I20260709 13:14:03.493958  3553 tablet_bootstrap.cc:492] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:03.494463  3553 ts_tablet_manager.cc:1403] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.121s	user 0.009s	sys 0.002s
I20260709 13:14:03.494776  3554 raft_consensus.cc:359] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.495482  3554 raft_consensus.cc:385] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.495764  3554 raft_consensus.cc:740] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.498008  3552 consensus_queue.cc:260] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.498584  3554 consensus_queue.cc:260] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.499550  3553 raft_consensus.cc:359] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.500226  3553 raft_consensus.cc:385] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.500480  3553 raft_consensus.cc:740] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.502089  3180 catalog_manager.cc:5697] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:03.501381  3554 ts_tablet_manager.cc:1434] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:14:03.503654  3552 ts_tablet_manager.cc:1434] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.027s	user 0.002s	sys 0.004s
I20260709 13:14:03.503996  3554 tablet_bootstrap.cc:492] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:03.511260  3554 tablet_bootstrap.cc:654] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.516361  3554 tablet_bootstrap.cc:492] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.516795  3554 ts_tablet_manager.cc:1403] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.013s	user 0.005s	sys 0.006s
I20260709 13:14:03.522751  3554 raft_consensus.cc:359] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.523500  3554 raft_consensus.cc:385] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.523810  3554 raft_consensus.cc:740] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.524186  3553 consensus_queue.cc:260] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.524554  3554 consensus_queue.cc:260] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.526626  3179 catalog_manager.cc:5697] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.529349  3554 ts_tablet_manager.cc:1434] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.004s
I20260709 13:14:03.530082  3553 ts_tablet_manager.cc:1434] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:14:03.531045  3553 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:03.504714  3552 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:03.534192  3554 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:03.540755  3554 tablet_bootstrap.cc:654] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.541484  3551 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:03.542012  3551 ts_tablet_manager.cc:1403] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.096s	user 0.010s	sys 0.000s
I20260709 13:14:03.544605  3551 raft_consensus.cc:359] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.545322  3551 raft_consensus.cc:385] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.562078  3554 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.562624  3554 ts_tablet_manager.cc:1403] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.005s
I20260709 13:14:03.564899  3553 tablet_bootstrap.cc:654] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.565891  3554 raft_consensus.cc:359] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.566607  3554 raft_consensus.cc:385] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.566887  3554 raft_consensus.cc:740] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.545574  3551 raft_consensus.cc:740] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.567965  3552 tablet_bootstrap.cc:654] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.567698  3554 consensus_queue.cc:260] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.568145  3551 consensus_queue.cc:260] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.570624  3551 ts_tablet_manager.cc:1434] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:14:03.571009  3554 ts_tablet_manager.cc:1434] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:14:03.572160  3554 tablet_bootstrap.cc:492] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:03.572319  3551 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:03.579437  3551 tablet_bootstrap.cc:654] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.583756  3553 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:03.584316  3553 ts_tablet_manager.cc:1403] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.001s
I20260709 13:14:03.584777  3554 tablet_bootstrap.cc:654] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.587049  3552 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:03.587553  3552 ts_tablet_manager.cc:1403] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.083s	user 0.011s	sys 0.001s
I20260709 13:14:03.587502  3553 raft_consensus.cc:359] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.589398  3553 raft_consensus.cc:385] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.589680  3553 raft_consensus.cc:740] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.594973  3552 raft_consensus.cc:359] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.595681  3552 raft_consensus.cc:385] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.595934  3552 raft_consensus.cc:740] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.596009  3861 raft_consensus.cc:493] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.596580  3553 consensus_queue.cc:260] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.596524  3861 raft_consensus.cc:515] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.596642  3552 consensus_queue.cc:260] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.598824  3861 leader_election.cc:290] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:03.598989  3552 ts_tablet_manager.cc:1434] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 13:14:03.600032  3552 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:03.600485  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85719491f2464bbf9c92b2f19238beeb" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:03.604260  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85719491f2464bbf9c92b2f19238beeb" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:03.604728  3428 raft_consensus.cc:2468] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:03.604883  3353 raft_consensus.cc:2468] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:03.606180  3470 leader_election.cc:304] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:03.607470  3861 raft_consensus.cc:2804] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.607818  3861 raft_consensus.cc:493] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.608176  3861 raft_consensus.cc:3060] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.612402  3553 ts_tablet_manager.cc:1434] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.004s
I20260709 13:14:03.615181  3552 tablet_bootstrap.cc:654] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.617398  3551 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:03.618211  3551 ts_tablet_manager.cc:1403] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.001s
I20260709 13:14:03.627565  3553 tablet_bootstrap.cc:492] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:03.629074  3503 raft_consensus.cc:3060] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.630115  3554 tablet_bootstrap.cc:492] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.630684  3554 ts_tablet_manager.cc:1403] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.005s
I20260709 13:14:03.632659  3551 raft_consensus.cc:359] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.633350  3551 raft_consensus.cc:385] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.633642  3551 raft_consensus.cc:740] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.633522  3554 raft_consensus.cc:359] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.634249  3554 raft_consensus.cc:385] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.634532  3554 raft_consensus.cc:740] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.634444  3551 consensus_queue.cc:260] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.634640  3553 tablet_bootstrap.cc:654] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.635318  3554 consensus_queue.cc:260] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.638268  3861 raft_consensus.cc:515] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.639151  3554 ts_tablet_manager.cc:1434] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:14:03.642530  3551 ts_tablet_manager.cc:1434] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 13:14:03.655517  3551 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea: Bootstrap starting.
I20260709 13:14:03.656075  3861 leader_election.cc:290] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:03.656644  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85719491f2464bbf9c92b2f19238beeb" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:03.657286  3353 raft_consensus.cc:3060] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.657429  3554 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:03.662112  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85719491f2464bbf9c92b2f19238beeb" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:03.662743  3428 raft_consensus.cc:3060] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.663195  3554 tablet_bootstrap.cc:654] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.651355  3552 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:03.666831  3552 ts_tablet_manager.cc:1403] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.007s
I20260709 13:14:03.669697  3552 raft_consensus.cc:359] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.670372  3552 raft_consensus.cc:385] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.670639  3552 raft_consensus.cc:740] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.672719  3428 raft_consensus.cc:2468] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:03.673972  3470 leader_election.cc:304] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:03.674044  3353 raft_consensus.cc:2468] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:03.674926  3861 raft_consensus.cc:2804] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:03.676518  3861 raft_consensus.cc:697] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:03.677680  3861 consensus_queue.cc:237] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.692260  3551 tablet_bootstrap.cc:654] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.697072  3554 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.697554  3554 ts_tablet_manager.cc:1403] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.040s	user 0.005s	sys 0.004s
I20260709 13:14:03.700703  3553 tablet_bootstrap.cc:492] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:03.701390  3553 ts_tablet_manager.cc:1403] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.074s	user 0.028s	sys 0.010s
I20260709 13:14:03.704789  3553 raft_consensus.cc:359] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.705672  3553 raft_consensus.cc:385] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.705958  3553 raft_consensus.cc:740] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.671259  3552 consensus_queue.cc:260] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.709975  3552 ts_tablet_manager.cc:1434] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.000s
I20260709 13:14:03.699748  3554 raft_consensus.cc:359] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.710690  3554 raft_consensus.cc:385] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.711119  3554 raft_consensus.cc:740] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.710523  3553 consensus_queue.cc:260] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.711808  3554 consensus_queue.cc:260] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.713407  3553 ts_tablet_manager.cc:1434] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:14:03.715440  3553 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:03.715945  3552 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:03.713860  3554 ts_tablet_manager.cc:1434] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:14:03.720314  3554 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:03.724208  3782 raft_consensus.cc:493] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.724787  3861 raft_consensus.cc:493] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.726524  3552 tablet_bootstrap.cc:654] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.732511  3782 raft_consensus.cc:515] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.734935  3782 leader_election.cc:290] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.732550  3861 raft_consensus.cc:515] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.736836  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:03.737426  3861 leader_election.cc:290] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:03.737512  3279 raft_consensus.cc:2468] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:14:03.728147  3551 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea: No bootstrap required, opened a new log
I20260709 13:14:03.728575  3874 raft_consensus.cc:493] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.739049  3551 ts_tablet_manager.cc:1403] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea: Time spent bootstrapping tablet: real 0.084s	user 0.007s	sys 0.004s
I20260709 13:14:03.739041  3874 raft_consensus.cc:515] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.741469  3874 leader_election.cc:290] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.742317  3551 raft_consensus.cc:359] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.742981  3551 raft_consensus.cc:385] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.743187  3553 tablet_bootstrap.cc:654] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.743269  3551 raft_consensus.cc:740] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.743435  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:03.739092  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:03.744108  3506 raft_consensus.cc:2468] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:03.751029  3249 leader_election.cc:304] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:03.739043  3397 leader_election.cc:304] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:03.744868  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:03.746342  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05be967181241fdb7f127f3505bbebc" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:03.753651  3782 raft_consensus.cc:2804] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.753823  3554 tablet_bootstrap.cc:654] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.754092  3782 raft_consensus.cc:493] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.754516  3782 raft_consensus.cc:3060] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:03.754647  3471 leader_election.cc:343] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.744032  3551 consensus_queue.cc:260] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.751858  3874 raft_consensus.cc:2804] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.757635  3551 ts_tablet_manager.cc:1434] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:14:03.757797  3874 raft_consensus.cc:493] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.758224  3874 raft_consensus.cc:3060] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.732539  3179 catalog_manager.cc:5697] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:03.753101  3396 leader_election.cc:343] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:03.772614  3247 leader_election.cc:343] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.774397  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05be967181241fdb7f127f3505bbebc" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:03.775612  3428 raft_consensus.cc:2468] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:03.784256  3874 raft_consensus.cc:515] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.785776  3470 leader_election.cc:304] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:03.786540  3861 raft_consensus.cc:2804] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.786891  3861 raft_consensus.cc:493] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.788664  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:03.789237  3506 raft_consensus.cc:3060] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.787264  3861 raft_consensus.cc:3060] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.784257  3782 raft_consensus.cc:515] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.794448  3554 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.794983  3554 ts_tablet_manager.cc:1403] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.075s	user 0.028s	sys 0.007s
I20260709 13:14:03.795732  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:03.796383  3281 raft_consensus.cc:3060] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.797933  3554 raft_consensus.cc:359] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.798686  3554 raft_consensus.cc:385] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.798959  3554 raft_consensus.cc:740] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.799772  3554 consensus_queue.cc:260] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.802901  3554 ts_tablet_manager.cc:1434] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:14:03.809227  3554 tablet_bootstrap.cc:492] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:03.812106  3506 raft_consensus.cc:2468] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:03.815372  3249 leader_election.cc:304] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:03.816215  3786 raft_consensus.cc:2804] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:03.816697  3786 raft_consensus.cc:697] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:03.817245  3554 tablet_bootstrap.cc:654] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.817461  3786 consensus_queue.cc:237] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.814110  3861 raft_consensus.cc:515] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.827824  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05be967181241fdb7f127f3505bbebc" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:03.828022  3552 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:03.827828  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:03.828502  3428 raft_consensus.cc:3060] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.829917  3874 leader_election.cc:290] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.831931  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05be967181241fdb7f127f3505bbebc" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
W20260709 13:14:03.832784  3247 leader_election.cc:343] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:03.834311  3471 leader_election.cc:343] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.834144  3931 consensus_queue.cc:1048] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:03.840150  3552 ts_tablet_manager.cc:1403] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.124s	user 0.023s	sys 0.014s
I20260709 13:14:03.840739  3945 raft_consensus.cc:493] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:03.843884  3945 raft_consensus.cc:515] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.844384  3552 raft_consensus.cc:359] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.845382  3552 raft_consensus.cc:385] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.845728  3552 raft_consensus.cc:740] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.848915  3860 raft_consensus.cc:493] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.849475  3860 raft_consensus.cc:515] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.851536  3860 leader_election.cc:290] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:03.852707  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:03.858078  3553 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:03.857952  3855 consensus_queue.cc:1048] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:03.858647  3553 ts_tablet_manager.cc:1403] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.143s	user 0.006s	sys 0.004s
I20260709 13:14:03.848932  3552 consensus_queue.cc:260] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.861766  3553 raft_consensus.cc:359] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.862430  3553 raft_consensus.cc:385] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.862715  3553 raft_consensus.cc:740] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.864199  3552 ts_tablet_manager.cc:1434] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:14:03.865885  3552 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:03.866078  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:03.866936  3503 raft_consensus.cc:2393] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 2: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:03.863461  3553 consensus_queue.cc:260] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.870586  3553 ts_tablet_manager.cc:1434] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:14:03.872642  3553 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:03.874168  3945 leader_election.cc:290] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:03.874991  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:03.875914  3279 raft_consensus.cc:2468] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:03.877065  3323 leader_election.cc:304] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:03.878353  3428 raft_consensus.cc:2468] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:03.803936  3281 raft_consensus.cc:2468] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:03.881439  3945 raft_consensus.cc:2804] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:03.882004  3945 raft_consensus.cc:493] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:03.882351  3945 raft_consensus.cc:3060] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:03.882786  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:03.884307  3667 raft_consensus.cc:493] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.884744  3667 raft_consensus.cc:515] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.885002  3470 leader_election.cc:304] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:03.885749  3860 raft_consensus.cc:2804] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:03.886821  3667 leader_election.cc:290] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:03.886533  3860 raft_consensus.cc:697] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:03.888105  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:03.888406  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:03.889216  3501 raft_consensus.cc:2393] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:14:03.890240  3782 leader_election.cc:290] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.892297  3943 raft_consensus.cc:493] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.892860  3943 raft_consensus.cc:515] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.893440  3324 leader_election.cc:304] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:03.894677  3667 raft_consensus.cc:3060] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.895599  3943 leader_election.cc:290] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:03.885777  3552 tablet_bootstrap.cc:654] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:03.897111  3860 consensus_queue.cc:237] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:03.884761  3397 leader_election.cc:304] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:03.897807  3855 raft_consensus.cc:493] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.900740  3553 tablet_bootstrap.cc:654] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:03.899441  3396 leader_election.cc:343] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.902107  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63dbef5a847a4451b6e4e46d15341334" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:03.902761  3353 raft_consensus.cc:2468] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:03.902882  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:03.903882  3428 raft_consensus.cc:2393] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:14:03.903585  3855 raft_consensus.cc:515] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.900403  3782 raft_consensus.cc:2804] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:03.905452  3782 raft_consensus.cc:697] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:03.906020  3855 leader_election.cc:290] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:03.906361  3782 consensus_queue.cc:237] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.908751  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:03.909296  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63dbef5a847a4451b6e4e46d15341334" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:03.909968  3428 raft_consensus.cc:2468] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
W20260709 13:14:03.910888  3247 leader_election.cc:343] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:03.910969  3470 leader_election.cc:304] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:03.912168  3943 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.912607  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:03.913335  3501 raft_consensus.cc:2468] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:03.917970  3179 catalog_manager.cc:5697] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:03.921969  3470 leader_election.cc:304] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:03.922750  3862 raft_consensus.cc:2804] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.923132  3862 raft_consensus.cc:493] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.923458  3862 raft_consensus.cc:3060] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.927610  3667 raft_consensus.cc:2749] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:03.932022  3249 leader_election.cc:304] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:03.935122  3874 raft_consensus.cc:2804] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:03.936120  3874 raft_consensus.cc:493] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.936494  3874 raft_consensus.cc:3060] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.901631  3861 leader_election.cc:290] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:03.940172  3554 tablet_bootstrap.cc:492] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:03.940723  3554 ts_tablet_manager.cc:1403] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.136s	user 0.027s	sys 0.062s
I20260709 13:14:03.942821  3945 raft_consensus.cc:515] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.946547  3862 raft_consensus.cc:515] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.949528  3554 raft_consensus.cc:359] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.951558  3554 raft_consensus.cc:385] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:03.951911  3554 raft_consensus.cc:740] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:03.952122  3943 raft_consensus.cc:2749] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:03.952694  3554 consensus_queue.cc:260] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:03.957016  3862 leader_election.cc:290] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:03.977859  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63dbef5a847a4451b6e4e46d15341334" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:03.986734  3356 raft_consensus.cc:3060] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:03.987015  3552 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:03.987743  3552 ts_tablet_manager.cc:1403] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.122s	user 0.010s	sys 0.015s
I20260709 13:14:03.971784  3927 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.975116  3895 raft_consensus.cc:493] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.961053  3874 raft_consensus.cc:515] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:03.979367  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63dbef5a847a4451b6e4e46d15341334" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:03.969270  3786 consensus_queue.cc:1048] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:14:04.006134  3552 raft_consensus.cc:359] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:04.009737  3552 raft_consensus.cc:385] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:04.010243  3552 raft_consensus.cc:740] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:04.012250  3181 catalog_manager.cc:5697] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.019935  3552 consensus_queue.cc:260] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:04.023406  3554 ts_tablet_manager.cc:1434] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.082s	user 0.023s	sys 0.008s
I20260709 13:14:04.015400  3428 raft_consensus.cc:3060] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:04.031594  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:04.036752  3506 raft_consensus.cc:2393] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 2: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:03.995909  3927 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:03.997594  3895 raft_consensus.cc:515] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:04.043298  3552 ts_tablet_manager.cc:1434] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.001s
I20260709 13:14:04.058480  3554 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
I20260709 13:14:04.060540  3552 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:04.070509  3927 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:04.073827  3553 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:04.074908  3553 ts_tablet_manager.cc:1403] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.202s	user 0.012s	sys 0.019s
I20260709 13:14:04.078344  3280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:04.060271  3945 leader_election.cc:290] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:04.085840  3180 catalog_manager.cc:5697] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:04.085608  3895 leader_election.cc:290] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:04.098234  3356 raft_consensus.cc:2468] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:04.100998  3874 leader_election.cc:290] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:04.114071  3280 raft_consensus.cc:2468] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:04.126129  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2794d9308d0458aa11c33cb7dab1748" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:04.133909  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:04.143009  3283 raft_consensus.cc:2393] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:14:04.143813  3554 tablet_bootstrap.cc:654] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:04.144071  3323 leader_election.cc:304] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:04.161471  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:04.164171  3552 tablet_bootstrap.cc:654] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:04.153213  3985 raft_consensus.cc:2804] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:04.140314  3279 raft_consensus.cc:2468] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 0.
I20260709 13:14:04.169071  3470 leader_election.cc:304] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:04.173075  3428 raft_consensus.cc:2468] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:04.179401  3899 raft_consensus.cc:2804] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:04.179910  3899 raft_consensus.cc:697] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:04.174479  3506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:04.180632  3506 raft_consensus.cc:3060] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:04.180791  3899 consensus_queue.cc:237] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:04.169363  3985 raft_consensus.cc:697] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:04.185123  3985 consensus_queue.cc:237] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:04.177109  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:04.188405  3430 raft_consensus.cc:2393] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 1: Already voted for candidate 06964934536148c2aa77a2f1c6382fea in this term.
I20260709 13:14:04.159574  3553 raft_consensus.cc:359] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:04.195475  3553 raft_consensus.cc:385] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:04.195849  3553 raft_consensus.cc:740] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:04.196637  3506 raft_consensus.cc:2468] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:04.198865  3397 leader_election.cc:304] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:04.197644  3249 leader_election.cc:304] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:04.195784  3861 raft_consensus.cc:493] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.213397  3861 raft_consensus.cc:515] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:04.215451  3861 leader_election.cc:290] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:04.216629  3874 raft_consensus.cc:2804] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:04.217195  3874 raft_consensus.cc:697] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:04.218039  3874 consensus_queue.cc:237] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:04.210486  3984 raft_consensus.cc:2804] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:04.219484  3984 raft_consensus.cc:493] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.224182  3553 consensus_queue.cc:260] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:04.219825  3984 raft_consensus.cc:3060] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:04.226804  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2794d9308d0458aa11c33cb7dab1748" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:04.228256  3353 raft_consensus.cc:3060] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:04.242280  3987 raft_consensus.cc:493] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.251614  3553 ts_tablet_manager.cc:1434] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.176s	user 0.011s	sys 0.000s
I20260709 13:14:04.255771  3987 raft_consensus.cc:515] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:04.249934  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:04.239277  3899 raft_consensus.cc:493] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:04.289024  3323 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 55326.2us
I20260709 13:14:04.267406  3976 consensus_queue.cc:1048] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260709 13:14:04.317404  3554 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:04.338716  3553 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
W20260709 13:14:04.301822  3398 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 65651.2us
I20260709 13:14:04.454880  3554 ts_tablet_manager.cc:1403] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.410s	user 0.022s	sys 0.013s
I20260709 13:14:04.428668  3181 catalog_manager.cc:5697] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.429697  3180 catalog_manager.cc:5697] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 2 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.289146  3786 raft_consensus.cc:493] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:04.467837  3470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 62823.1us
I20260709 13:14:04.291010  3427 raft_consensus.cc:2468] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:04.471591  3552 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:04.472172  3552 ts_tablet_manager.cc:1403] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.425s	user 0.045s	sys 0.049s
W20260709 13:14:04.476910  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:04.315730  3987 leader_election.cc:290] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:04.337563  3355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:04.388974  3322 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
W20260709 13:14:04.489843  3322 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 100972us
I20260709 13:14:04.491321  3945 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:04.429318  3984 raft_consensus.cc:515] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:04.454823  3980 consensus_queue.cc:1048] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 13:14:04.499434  3945 raft_consensus.cc:2749] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:04.287495  3899 raft_consensus.cc:515] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
W20260709 13:14:04.507862  3470 leader_election.cc:343] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:04.397786  3396 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 148071us
I20260709 13:14:04.481784  3943 consensus_queue.cc:1048] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:04.379725  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a384f005d84258a11b79619f7d7ef4" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:04.498265  3552 raft_consensus.cc:359] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:04.510933  3552 raft_consensus.cc:385] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:04.511327  3552 raft_consensus.cc:740] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:04.324906  3997 consensus_queue.cc:1048] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.164s
I20260709 13:14:04.342718  3353 raft_consensus.cc:2468] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:04.520908  3552 consensus_queue.cc:260] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:04.528642  3554 raft_consensus.cc:359] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:04.531039  3554 raft_consensus.cc:385] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:04.531337  3554 raft_consensus.cc:740] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:04.473631  3786 raft_consensus.cc:515] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:04.532684  3554 consensus_queue.cc:260] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
W20260709 13:14:04.534427  3249 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 60084.8us
I20260709 13:14:04.545914  3554 ts_tablet_manager.cc:1434] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.090s	user 0.004s	sys 0.003s
I20260709 13:14:04.547222  3554 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f: Bootstrap starting.
W20260709 13:14:04.556890  3396 leader_election.cc:343] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:04.559633  3470 leader_election.cc:304] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:04.541800  3984 leader_election.cc:290] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:04.549322  3279 raft_consensus.cc:3060] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:04.566184  3397 leader_election.cc:343] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:04.570809  3786 leader_election.cc:290] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:04.570040  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2794d9308d0458aa11c33cb7dab1748" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:04.562690  3862 raft_consensus.cc:2804] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:04.573175  3283 raft_consensus.cc:3060] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:04.573455  3862 raft_consensus.cc:493] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.573846  3862 raft_consensus.cc:3060] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:04.565673  3943 consensus_queue.cc:1048] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:14:04.582182  3553 tablet_bootstrap.cc:654] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:04.583495  3283 raft_consensus.cc:2468] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:04.587033  3977 consensus_queue.cc:1048] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:04.587972  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a384f005d84258a11b79619f7d7ef4" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:04.588197  3899 leader_election.cc:290] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:04.591912  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:04.593734  3283 raft_consensus.cc:2410] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:14:04.594806  3862 raft_consensus.cc:515] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:04.610693  3862 leader_election.cc:290] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:04.613075  3552 ts_tablet_manager.cc:1434] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.141s	user 0.011s	sys 0.012s
W20260709 13:14:04.683621  3470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 66528.7us
I20260709 13:14:04.684523  3397 leader_election.cc:304] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:04.687949  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:04.688555  3432 raft_consensus.cc:3060] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:04.690197  3552 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:04.690881  3554 tablet_bootstrap.cc:654] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:04.696317  3249 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 87776.1us
I20260709 13:14:04.695709  3357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:04.689462  3895 raft_consensus.cc:2804] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:14:04.726172  3470 leader_election.cc:343] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:04.737936  3432 raft_consensus.cc:2468] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:04.739125  3470 leader_election.cc:304] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:04.724452  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05be967181241fdb7f127f3505bbebc" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:04.743851  4003 raft_consensus.cc:2804] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:04.760409  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05be967181241fdb7f127f3505bbebc" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
W20260709 13:14:04.762290  3398 leader_election.cc:343] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:04.762758  3398 leader_election.cc:304] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:04.766127  3281 raft_consensus.cc:1275] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:04.764449  3999 raft_consensus.cc:2749] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:04.769023  3786 consensus_queue.cc:1048] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:04.768863  3249 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 61989.7us
I20260709 13:14:04.734409  3895 raft_consensus.cc:697] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:04.781677  3895 consensus_queue.cc:237] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:04.772751  3353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2794d9308d0458aa11c33cb7dab1748" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:04.770136  3927 consensus_queue.cc:1048] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:04.799484  3247 leader_election.cc:304] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972a4469a4654416ab0a153d8c740282; no voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:04.761947  4003 raft_consensus.cc:697] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:04.801817  3899 raft_consensus.cc:493] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.819422  3553 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:04.820341  3553 ts_tablet_manager.cc:1403] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.482s	user 0.020s	sys 0.019s
I20260709 13:14:04.807390  3931 raft_consensus.cc:2749] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:04.823671  3396 consensus_peers.cc:597] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:04.827912  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094d0615636b4da4a2c8ba92b4218624" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:04.822808  3899 raft_consensus.cc:515] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
W20260709 13:14:04.904258  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 74033.5us
I20260709 13:14:04.921666  3554 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f: No bootstrap required, opened a new log
I20260709 13:14:04.922181  3554 ts_tablet_manager.cc:1403] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent bootstrapping tablet: real 0.375s	user 0.017s	sys 0.009s
I20260709 13:14:04.924129  4013 raft_consensus.cc:493] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.924662  4013 raft_consensus.cc:515] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:04.926760  4013 leader_election.cc:290] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
W20260709 13:14:04.900774  3396 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 79091.1us
I20260709 13:14:04.932960  3354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:04.939185  3505 raft_consensus.cc:1275] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:04.829696  3862 consensus_queue.cc:1048] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.089s
I20260709 13:14:04.942006  4003 consensus_queue.cc:237] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:04.944159  3899 leader_election.cc:290] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
W20260709 13:14:04.948177  3396 leader_election.cc:343] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:04.948580  3179 catalog_manager.cc:5697] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:04.949433  3553 raft_consensus.cc:359] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:04.950096  3553 raft_consensus.cc:385] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:04.950340  3553 raft_consensus.cc:740] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:04.950913  3553 consensus_queue.cc:260] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:04.953387  3553 ts_tablet_manager.cc:1434] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.133s	user 0.004s	sys 0.002s
I20260709 13:14:04.954370  3553 tablet_bootstrap.cc:492] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:04.956806  3874 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.935942  3552 tablet_bootstrap.cc:654] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:04.958350  4009 consensus_queue.cc:1048] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:14:04.976186  4012 consensus_queue.cc:1048] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:04.967042  3874 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:04.985410  3874 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
W20260709 13:14:04.973014  3247 leader_election.cc:343] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:04.987479  3553 tablet_bootstrap.cc:654] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:04.989604  3470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 68800.3us
I20260709 13:14:04.992884  3554 raft_consensus.cc:359] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:04.993619  3554 raft_consensus.cc:385] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:04.993984  3554 raft_consensus.cc:740] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Initialized, Role: FOLLOWER
I20260709 13:14:04.994822  3554 consensus_queue.cc:260] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:05.012850  3470 leader_election.cc:304] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:04.997324  3985 raft_consensus.cc:493] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.047290  3554 ts_tablet_manager.cc:1434] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f: Time spent starting tablet: real 0.125s	user 0.007s	sys 0.005s
W20260709 13:14:05.054018  3323 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 56471.3us
I20260709 13:14:05.055868  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:05.056542  3431 raft_consensus.cc:2468] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:05.018071  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:05.061802  3247 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:05.068315  3356 raft_consensus.cc:2468] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:05.071054  3977 raft_consensus.cc:2804] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:05.071528  3977 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.071851  3977 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:05.017498  3985 raft_consensus.cc:515] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:05.076174  3985 leader_election.cc:290] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
W20260709 13:14:05.092334  3470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 79599us
I20260709 13:14:05.099813  3552 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: No bootstrap required, opened a new log
I20260709 13:14:05.100644  3552 ts_tablet_manager.cc:1403] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.425s	user 0.016s	sys 0.013s
I20260709 13:14:05.089241  3351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:05.101790  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:05.097661  4008 raft_consensus.cc:2749] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:05.128070  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:05.128842  3502 raft_consensus.cc:2468] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 0.
I20260709 13:14:05.130079  3355 raft_consensus.cc:3060] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:05.140213  3249 leader_election.cc:304] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:05.155093  4010 consensus_queue.cc:1048] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:05.156183  3278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:05.159662  3988 raft_consensus.cc:493] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.160590  3988 raft_consensus.cc:515] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:05.155444  3977 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:14:05.167435  3470 leader_election.cc:343] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:05.175221  4034 consensus_queue.cc:1048] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.049s
I20260709 13:14:05.149308  3874 raft_consensus.cc:2804] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:05.176455  3553 tablet_bootstrap.cc:492] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:05.177007  3553 ts_tablet_manager.cc:1403] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.223s	user 0.012s	sys 0.028s
I20260709 13:14:05.179701  3552 raft_consensus.cc:359] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:05.180300  3552 raft_consensus.cc:385] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:05.180586  3552 raft_consensus.cc:740] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:05.181241  3552 consensus_queue.cc:260] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:05.202760  3860 consensus_queue.cc:1048] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:14:05.204540  3281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:05.179365  3874 raft_consensus.cc:493] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.205320  3874 raft_consensus.cc:3060] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:05.247080  3552 ts_tablet_manager.cc:1434] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.146s	user 0.009s	sys 0.005s
I20260709 13:14:05.251372  3553 raft_consensus.cc:359] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:05.252354  3553 raft_consensus.cc:385] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:05.252641  3553 raft_consensus.cc:740] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:05.254736  3432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a384f005d84258a11b79619f7d7ef4" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:05.255484  3432 raft_consensus.cc:2468] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:05.256500  3470 leader_election.cc:304] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:05.257751  4026 raft_consensus.cc:2804] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:05.258240  4026 raft_consensus.cc:493] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.258639  4026 raft_consensus.cc:3060] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:05.260246  3988 leader_election.cc:290] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:05.263134  3977 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:05.266366  3277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a384f005d84258a11b79619f7d7ef4" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:05.267114  3277 raft_consensus.cc:2468] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 0.
I20260709 13:14:05.270325  3553 consensus_queue.cc:260] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:05.290486  4026 raft_consensus.cc:515] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:05.298918  4026 leader_election.cc:290] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:05.306247  3283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a384f005d84258a11b79619f7d7ef4" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:05.306967  3283 raft_consensus.cc:3060] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:05.321134  3874 raft_consensus.cc:515] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:05.324486  3553 ts_tablet_manager.cc:1434] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.147s	user 0.008s	sys 0.000s
I20260709 13:14:05.326889  3428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a384f005d84258a11b79619f7d7ef4" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:05.330050  3428 raft_consensus.cc:3060] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:05.341344  3281 raft_consensus.cc:2393] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 1: Already voted for candidate 972a4469a4654416ab0a153d8c740282 in this term.
I20260709 13:14:05.343086  3283 raft_consensus.cc:2468] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:05.345849  3428 raft_consensus.cc:2468] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:05.345983  3874 leader_election.cc:290] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:05.347482  3553 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
W20260709 13:14:05.349509  3396 consensus_peers.cc:597] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:05.357959  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:05.358625  3501 raft_consensus.cc:3060] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:05.355351  3471 leader_election.cc:304] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:05.368687  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:05.369320  3352 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:05.371994  3356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:05.377403  3322 leader_election.cc:304] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:05.384843  3471 leader_election.cc:304] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:05.385099  3429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:05.393674  3429 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:05.386452  3994 raft_consensus.cc:2749] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:05.386088  3988 raft_consensus.cc:2804] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:05.396446  3988 raft_consensus.cc:697] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:05.397294  3988 consensus_queue.cc:237] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:05.399477  3352 raft_consensus.cc:2468] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
W20260709 13:14:05.410907  3247 leader_election.cc:343] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:05.413338  3247 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:05.440665  3999 consensus_queue.cc:1048] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:05.455726  3874 raft_consensus.cc:2804] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:05.456722  3874 raft_consensus.cc:697] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:05.457834  3874 consensus_queue.cc:237] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:05.461623  3501 raft_consensus.cc:2468] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 1.
I20260709 13:14:05.462746  4008 raft_consensus.cc:2749] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:05.465771  3249 leader_election.cc:304] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:05.466960  4009 raft_consensus.cc:2804] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:05.467595  4009 raft_consensus.cc:697] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
W20260709 13:14:05.469254  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 56012.8us
I20260709 13:14:05.452713  3179 catalog_manager.cc:5697] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 1 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:05.481217  4009 consensus_queue.cc:237] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:05.485201  3553 tablet_bootstrap.cc:654] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:05.530580  3429 raft_consensus.cc:2468] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
W20260709 13:14:05.573467  3229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:14:05.693914  4047 log.cc:927] Time spent T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:14:05.697047  3471 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 101975us
I20260709 13:14:05.814625  3993 raft_consensus.cc:493] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:05.856423  4046 log.cc:927] Time spent T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:14:05.868103  3993 raft_consensus.cc:515] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:05.872215  3993 leader_election.cc:290] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
W20260709 13:14:05.873117  4052 log.cc:927] Time spent T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Append to log took a long time: real 0.059s	user 0.000s	sys 0.002s
I20260709 13:14:05.895279  3355 raft_consensus.cc:3060] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:05.905216  3396 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 208344us
I20260709 13:14:05.965667  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:14:05.979349  3247 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.66:33623, user_credentials={real_user=slave}} blocked reactor thread for 67631.9us
I20260709 13:14:05.990471  4056 consensus_queue.cc:1048] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:14:06.009009  3553 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:06.009559  3553 ts_tablet_manager.cc:1403] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.662s	user 0.016s	sys 0.024s
I20260709 13:14:06.012921  3553 raft_consensus.cc:359] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:06.013635  3553 raft_consensus.cc:385] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:06.013899  3553 raft_consensus.cc:740] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:06.014722  3553 consensus_queue.cc:260] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:06.020331  3553 ts_tablet_manager.cc:1434] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:14:06.022521  3553 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2: Bootstrap starting.
I20260709 13:14:06.031997  4026 consensus_queue.cc:1048] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:06.052650  3553 tablet_bootstrap.cc:654] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:06.055729  3976 consensus_queue.cc:1048] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:06.080453  3356 raft_consensus.cc:3060] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:06.098284  3350 raft_consensus.cc:1275] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:06.103986  3978 consensus_queue.cc:1048] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:06.111433  3978 consensus_queue.cc:1048] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:06.141889  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:06.136348  4009 consensus_queue.cc:1048] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:14:06.161998  3427 raft_consensus.cc:1275] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:06.186092  3976 consensus_queue.cc:1048] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:06.206908  3322 leader_election.cc:304] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:06.212530  3993 raft_consensus.cc:2749] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:06.223371  3553 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2: No bootstrap required, opened a new log
I20260709 13:14:06.237038  3553 ts_tablet_manager.cc:1403] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2: Time spent bootstrapping tablet: real 0.215s	user 0.010s	sys 0.008s
I20260709 13:14:06.241006  3553 raft_consensus.cc:359] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:06.241765  3553 raft_consensus.cc:385] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:06.242097  3553 raft_consensus.cc:740] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:06.267037  3978 consensus_queue.cc:1048] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:14:06.282238  3553 consensus_queue.cc:260] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:06.298362  3553 ts_tablet_manager.cc:1434] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
I20260709 13:14:06.307718  3178 catalog_manager.cc:5697] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:06.309774  3178 catalog_manager.cc:5697] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 1 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:06.348981  3180 catalog_manager.cc:5697] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 0 to 2, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } }
I20260709 13:14:06.365759  3181 catalog_manager.cc:5697] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:06.495512  3667 raft_consensus.cc:493] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.496021  3667 raft_consensus.cc:515] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:06.497953  3667 leader_election.cc:290] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:06.499723  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2794d9308d0458aa11c33cb7dab1748" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:06.510170  3279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2794d9308d0458aa11c33cb7dab1748" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:06.511333  3323 leader_election.cc:304] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:06.512023  3985 raft_consensus.cc:2749] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:06.524981  3358 raft_consensus.cc:3060] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:07.630048 27901 tablet_server.cc:179] TabletServer@127.27.63.65:0 shutting down...
I20260709 13:14:07.678944 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:07.679852 27901 tablet_replica.cc:333] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.681090 27901 raft_consensus.cc:2243] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.681634 27901 raft_consensus.cc:2272] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.683781 27901 tablet_replica.cc:333] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.684510 27901 raft_consensus.cc:2243] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.686020 27901 raft_consensus.cc:2272] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.688385 27901 tablet_replica.cc:333] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
W20260709 13:14:07.688804  3397 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:07.689133 27901 raft_consensus.cc:2243] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.690101 27901 raft_consensus.cc:2272] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.693585 27901 tablet_replica.cc:333] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.694392 27901 raft_consensus.cc:2243] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.703117 27901 raft_consensus.cc:2272] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.704640  3397 consensus_peers.cc:597] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.709311 27901 tablet_replica.cc:333] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
W20260709 13:14:07.709461  3471 consensus_peers.cc:597] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.710331 27901 raft_consensus.cc:2243] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.710840 27901 raft_consensus.cc:2272] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.711719  3471 consensus_peers.cc:597] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.713016 27901 tablet_replica.cc:333] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.713635 27901 raft_consensus.cc:2243] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.714054 27901 raft_consensus.cc:2272] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.715904 27901 tablet_replica.cc:333] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.716490 27901 raft_consensus.cc:2243] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.717182 27901 raft_consensus.cc:2272] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.719110 27901 tablet_replica.cc:333] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.719811 27901 raft_consensus.cc:2243] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.720286 27901 raft_consensus.cc:2272] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.722536 27901 tablet_replica.cc:333] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.723076 27901 raft_consensus.cc:2243] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.723872 27901 raft_consensus.cc:2272] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.726046 27901 tablet_replica.cc:333] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.726752 27901 raft_consensus.cc:2243] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.727293 27901 raft_consensus.cc:2272] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.729719  3471 consensus_peers.cc:597] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.730027 27901 tablet_replica.cc:333] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.730739 27901 raft_consensus.cc:2243] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.731215 27901 raft_consensus.cc:2272] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.733063 27901 tablet_replica.cc:333] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.733592 27901 raft_consensus.cc:2243] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.734072 27901 raft_consensus.cc:2272] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.736181 27901 tablet_replica.cc:333] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.736753 27901 raft_consensus.cc:2243] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.737560 27901 raft_consensus.cc:2272] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.738147  3323 consensus_peers.cc:597] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.739846 27901 tablet_replica.cc:333] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.740599 27901 raft_consensus.cc:2243] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.741531 27901 raft_consensus.cc:2272] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.742141  3397 consensus_peers.cc:597] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.743537 27901 tablet_replica.cc:333] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.744259 27901 raft_consensus.cc:2243] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.744684 27901 raft_consensus.cc:2272] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.746752 27901 tablet_replica.cc:333] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.747339 27901 raft_consensus.cc:2243] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.747731 27901 raft_consensus.cc:2272] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.749567 27901 tablet_replica.cc:333] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.750123 27901 raft_consensus.cc:2243] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.750903 27901 raft_consensus.cc:2272] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.752962 27901 tablet_replica.cc:333] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.753693 27901 raft_consensus.cc:2243] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.754165 27901 raft_consensus.cc:2272] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.756420 27901 tablet_replica.cc:333] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.757079 27901 raft_consensus.cc:2243] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.757913 27901 raft_consensus.cc:2272] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.760221 27901 tablet_replica.cc:333] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.760702 27901 raft_consensus.cc:2243] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.761060 27901 raft_consensus.cc:2272] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.762639 27901 tablet_replica.cc:333] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.763108 27901 raft_consensus.cc:2243] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.763773 27901 raft_consensus.cc:2272] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.765463 27901 tablet_replica.cc:333] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.765945 27901 raft_consensus.cc:2243] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.766397 27901 raft_consensus.cc:2272] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.768229 27901 tablet_replica.cc:333] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.768704 27901 raft_consensus.cc:2243] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.769037 27901 raft_consensus.cc:2272] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.770673 27901 tablet_replica.cc:333] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.771306 27901 raft_consensus.cc:2243] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.771775 27901 raft_consensus.cc:2272] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.773784  3397 consensus_peers.cc:597] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.774173 27901 tablet_replica.cc:333] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.774821 27901 raft_consensus.cc:2243] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.775697 27901 raft_consensus.cc:2272] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.778053 27901 tablet_replica.cc:333] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.778532 27901 raft_consensus.cc:2243] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.779191 27901 raft_consensus.cc:2272] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.781369 27901 tablet_replica.cc:333] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.781934 27901 raft_consensus.cc:2243] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.782775 27901 raft_consensus.cc:2272] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.784584 27901 tablet_replica.cc:333] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.785132 27901 raft_consensus.cc:2243] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.785558 27901 raft_consensus.cc:2272] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.787532 27901 tablet_replica.cc:333] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.788158 27901 raft_consensus.cc:2243] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.788563 27901 raft_consensus.cc:2272] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.790373 27901 tablet_replica.cc:333] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.790908 27901 raft_consensus.cc:2243] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.791297 27901 raft_consensus.cc:2272] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.793114 27901 tablet_replica.cc:333] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.793635 27901 raft_consensus.cc:2243] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.794021 27901 raft_consensus.cc:2272] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.795852 27901 tablet_replica.cc:333] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.796438 27901 raft_consensus.cc:2243] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.796838 27901 raft_consensus.cc:2272] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.798589 27901 tablet_replica.cc:333] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.799307 27901 raft_consensus.cc:2243] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.799826 27901 raft_consensus.cc:2272] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.802397 27901 tablet_replica.cc:333] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.803260 27901 raft_consensus.cc:2243] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.804301 27901 raft_consensus.cc:2272] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.806327  3397 consensus_peers.cc:597] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.806736 27901 tablet_replica.cc:333] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.807463 27901 raft_consensus.cc:2243] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.808254 27901 raft_consensus.cc:2272] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.810297 27901 tablet_replica.cc:333] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.810851 27901 raft_consensus.cc:2243] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.811825 27901 raft_consensus.cc:2272] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.813961 27901 tablet_replica.cc:333] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.814461 27901 raft_consensus.cc:2243] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.815203 27901 raft_consensus.cc:2272] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.817160 27901 tablet_replica.cc:333] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.817667 27901 raft_consensus.cc:2243] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.818070 27901 raft_consensus.cc:2272] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.819938 27901 tablet_replica.cc:333] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.820436 27901 raft_consensus.cc:2243] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.821166 27901 raft_consensus.cc:2272] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.822948 27901 tablet_replica.cc:333] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.823454 27901 raft_consensus.cc:2243] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.823849 27901 raft_consensus.cc:2272] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.826047 27901 tablet_replica.cc:333] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.826535 27901 raft_consensus.cc:2243] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.826984 27901 raft_consensus.cc:2272] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.829079 27901 tablet_replica.cc:333] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.829559 27901 raft_consensus.cc:2243] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.829938 27901 raft_consensus.cc:2272] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.832022 27901 tablet_replica.cc:333] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.832568 27901 raft_consensus.cc:2243] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.833525 27901 raft_consensus.cc:2272] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.835424  3471 consensus_peers.cc:597] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.836216 27901 tablet_replica.cc:333] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.836931 27901 raft_consensus.cc:2243] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.837473 27901 raft_consensus.cc:2272] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.839756 27901 tablet_replica.cc:333] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: stopping tablet replica
I20260709 13:14:07.840273 27901 raft_consensus.cc:2243] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.841107 27901 raft_consensus.cc:2272] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.842283  3323 consensus_peers.cc:597] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:07.857252  3323 consensus_peers.cc:597] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:07.873409  3471 consensus_peers.cc:597] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.913632 27901 tablet_server.cc:196] TabletServer@127.27.63.65:0 shutdown complete.
W20260709 13:14:07.945875  3471 consensus_peers.cc:597] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:07.952981  3397 consensus_peers.cc:597] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:07.967999  3397 consensus_peers.cc:597] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.988888 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:07.995102  3471 consensus_peers.cc:597] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:08.002168  4086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:08.003738  4087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:08.015236 27901 server_base.cc:1061] running on GCE node
W20260709 13:14:08.016700  3397 consensus_peers.cc:597] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:08.017820  4089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:08.020975 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:08.021314 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:08.021592 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602848021573 us; error 0 us; skew 500 ppm
I20260709 13:14:08.022136 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:08.026760 27901 webserver.cc:533] Webserver started at http://127.27.63.65:35109/ using document root <none> and password file <none>
W20260709 13:14:08.027102  3323 consensus_peers.cc:597] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:08.027415 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:08.027762 27901 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:14:08.032387 27901 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.001s
W20260709 13:14:08.034479  3397 consensus_peers.cc:597] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:08.040416  3323 consensus_peers.cc:597] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:08.043460  4095 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
W20260709 13:14:08.043521  3323 consensus_peers.cc:597] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:08.044713 27901 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:14:08.045104 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "972a4469a4654416ab0a153d8c740282"
format_stamp: "Formatted at 2026-07-09 13:13:54 on dist-test-slave-sjgx"
I20260709 13:14:08.045567 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:08.057417 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:08.058765 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:08.060302 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:08.064702  4102 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:14:08.093641  3397 consensus_peers.cc:597] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:08.125589  3471 consensus_peers.cc:597] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:08.127555  3397 consensus_peers.cc:597] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:08.139751  3323 consensus_peers.cc:597] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:08.190522  3397 consensus_peers.cc:597] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:08.298127 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:14:08.298458 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.235s	user 0.007s	sys 0.001s
I20260709 13:14:08.298705 27901 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:14:08.304006  4102 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.316313  4102 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.317099  4102 tablet_bootstrap.cc:492] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.317632  4102 ts_tablet_manager.cc:1403] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.014s	user 0.004s	sys 0.008s
I20260709 13:14:08.319561  4102 raft_consensus.cc:359] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.320026  4102 raft_consensus.cc:740] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.320729  4102 consensus_queue.cc:260] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.342583  4102 ts_tablet_manager.cc:1434] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.025s	user 0.017s	sys 0.008s
I20260709 13:14:08.343431  4102 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.358317  4102 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.359133  4102 tablet_bootstrap.cc:492] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.359645  4102 ts_tablet_manager.cc:1403] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 13:14:08.361477  4102 raft_consensus.cc:359] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.361970  4102 raft_consensus.cc:740] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.362537  4102 consensus_queue.cc:260] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.364480  4102 ts_tablet_manager.cc:1434] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:14:08.365203  4102 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.381963  4102 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.382679  4102 tablet_bootstrap.cc:492] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.383263  4102 ts_tablet_manager.cc:1403] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 13:14:08.385110  4102 raft_consensus.cc:359] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.385643  4102 raft_consensus.cc:740] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.386322  4102 consensus_queue.cc:260] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.388424  4102 ts_tablet_manager.cc:1434] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:14:08.389209  4102 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.444923  4102 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.445945  4102 tablet_bootstrap.cc:492] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.446632  4102 ts_tablet_manager.cc:1403] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.058s	user 0.033s	sys 0.023s
I20260709 13:14:08.449393  4102 raft_consensus.cc:359] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.450080  4102 raft_consensus.cc:740] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.450922  4102 consensus_queue.cc:260] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.452786  4102 ts_tablet_manager.cc:1434] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:08.453583  4102 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.467396  4102 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.468161  4102 tablet_bootstrap.cc:492] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.468650  4102 ts_tablet_manager.cc:1403] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 13:14:08.470393  4102 raft_consensus.cc:359] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.471067  4102 raft_consensus.cc:740] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.471794  4102 consensus_queue.cc:260] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.473626  4102 ts_tablet_manager.cc:1434] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:08.474200  4102 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.490090  4102 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.490864  4102 tablet_bootstrap.cc:492] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.491402  4102 ts_tablet_manager.cc:1403] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 13:14:08.493093  4102 raft_consensus.cc:359] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.493573  4102 raft_consensus.cc:740] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.494096  4102 consensus_queue.cc:260] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.495517  4102 ts_tablet_manager.cc:1434] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:14:08.496157  4102 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.506887  4102 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.507573  4102 tablet_bootstrap.cc:492] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.508036  4102 ts_tablet_manager.cc:1403] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:14:08.509733  4102 raft_consensus.cc:359] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.510258  4102 raft_consensus.cc:740] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.510759  4102 consensus_queue.cc:260] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.512120  4102 ts_tablet_manager.cc:1434] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:08.512715  4102 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.524199  4102 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.524878  4102 tablet_bootstrap.cc:492] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.525352  4102 ts_tablet_manager.cc:1403] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:14:08.527047  4102 raft_consensus.cc:359] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.527515  4102 raft_consensus.cc:740] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.528010  4102 consensus_queue.cc:260] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.529273  4102 ts_tablet_manager.cc:1434] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:08.529855  4102 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.574568 27901 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:14:08.574965 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.276s	user 0.199s	sys 0.006s
I20260709 13:14:08.609933  4102 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.611042  4102 tablet_bootstrap.cc:492] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.611716  4102 ts_tablet_manager.cc:1403] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.082s	user 0.025s	sys 0.042s
I20260709 13:14:08.614645  4102 raft_consensus.cc:359] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.615382  4102 raft_consensus.cc:740] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.616412  4102 consensus_queue.cc:260] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.620682  4102 ts_tablet_manager.cc:1434] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:14:08.621573  4102 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.643343  4102 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.644481  4102 tablet_bootstrap.cc:492] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.645290  4102 ts_tablet_manager.cc:1403] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 13:14:08.648520  4102 raft_consensus.cc:359] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.649359  4102 raft_consensus.cc:740] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.650357  4102 consensus_queue.cc:260] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.653033  4102 ts_tablet_manager.cc:1434] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:14:08.653975  4102 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.671747  4102 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.672977  4102 tablet_bootstrap.cc:492] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.673873  4102 ts_tablet_manager.cc:1403] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:14:08.677047  4102 raft_consensus.cc:359] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.677956  4102 raft_consensus.cc:740] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.678803  4102 consensus_queue.cc:260] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.681030  4102 ts_tablet_manager.cc:1434] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:08.681964  4102 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.684173 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:34091
I20260709 13:14:08.684298  4176 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:34091 every 8 connection(s)
I20260709 13:14:08.700243  4102 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.701040  4102 tablet_bootstrap.cc:492] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.701589  4102 ts_tablet_manager.cc:1403] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 13:14:08.703480  4107 raft_consensus.cc:493] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.704175  4107 raft_consensus.cc:515] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.704212  4102 raft_consensus.cc:359] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.705152  4102 raft_consensus.cc:740] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.706125  4102 consensus_queue.cc:260] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.706700  4107 leader_election.cc:290] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.708217  4102 ts_tablet_manager.cc:1434] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:14:08.709051  4102 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.710914  4177 heartbeater.cc:344] Connected to a master server at 127.27.63.126:36891
I20260709 13:14:08.711333  4177 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:08.712226  4177 heartbeater.cc:507] Master 127.27.63.126:36891 requested a full tablet report, sending...
I20260709 13:14:08.716615  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:08.729804  4079 raft_consensus.cc:493] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.730402  4079 raft_consensus.cc:515] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.732921  4079 leader_election.cc:290] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:08.737738  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:08.743003  3985 raft_consensus.cc:493] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.743789  3985 raft_consensus.cc:515] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.746426  3985 leader_election.cc:290] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:08.750345  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:08.752125  3179 ts_manager.cc:194] Re-registered known tserver with Master: 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.768030  4102 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:08.769098  4102 tablet_bootstrap.cc:492] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.770171  4102 ts_tablet_manager.cc:1403] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.061s	user 0.034s	sys 0.024s
I20260709 13:14:08.772975  4102 raft_consensus.cc:359] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.773684  4102 raft_consensus.cc:740] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.774328  4102 consensus_queue.cc:260] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.775142  4103 raft_consensus.cc:493] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.775269  4085 raft_consensus.cc:493] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.775847  4085 raft_consensus.cc:515] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.776767  4102 ts_tablet_manager.cc:1434] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:08.777642  4102 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.775734  4103 raft_consensus.cc:515] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.779552  4103 leader_election.cc:290] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.780609  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:08.782146  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:08.782758  3430 raft_consensus.cc:2468] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:08.784109  4085 leader_election.cc:290] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.787700  3985 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.788311  3985 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.790479  3985 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:08.791792  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:08.798666  4105 raft_consensus.cc:493] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.799235  4105 raft_consensus.cc:515] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.802595  3470 leader_election.cc:304] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:08.803107  4105 leader_election.cc:290] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:08.804822  4106 raft_consensus.cc:2804] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:08.805307  4106 raft_consensus.cc:493] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.805778  4106 raft_consensus.cc:3060] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:08.810583  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.813102  4107 raft_consensus.cc:493] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.813542  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.813735  4107 raft_consensus.cc:515] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.816576  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
W20260709 13:14:08.817209  3471 leader_election.cc:343] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:08.818602  3471 leader_election.cc:343] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.818987  3471 leader_election.cc:304] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.820425  4079 raft_consensus.cc:2749] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.822180  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:08.827569  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.829079  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:14:08.832788  3323 leader_election.cc:343] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.833861  3323 leader_election.cc:304] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.834849  3985 raft_consensus.cc:2749] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.835166  4107 leader_election.cc:290] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.839593  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.843056  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.844588  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.845238  4150 raft_consensus.cc:2468] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:08.847470  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:14:08.850543  3397 leader_election.cc:343] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.850979  3397 leader_election.cc:304] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.851655  4107 raft_consensus.cc:2749] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.852994  4104 raft_consensus.cc:493] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.853523  4104 raft_consensus.cc:515] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.855357  4104 leader_election.cc:290] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.855998  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:08.855638  4102 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:14:08.856874  3323 leader_election.cc:343] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.857108  3179 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38974
I20260709 13:14:08.857374  4102 tablet_bootstrap.cc:492] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:08.858116  4102 ts_tablet_manager.cc:1403] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.081s	user 0.033s	sys 0.027s
W20260709 13:14:08.859725  3397 leader_election.cc:343] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.860443  3397 leader_election.cc:304] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.861471  3397 leader_election.cc:304] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:08.861654  4102 raft_consensus.cc:359] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.862505  4102 raft_consensus.cc:740] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
W20260709 13:14:08.862774  3397 leader_election.cc:343] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.857322  3323 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.863137  4102 consensus_queue.cc:260] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.864027  4104 raft_consensus.cc:2749] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.861593  4107 raft_consensus.cc:2749] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.864847  4107 raft_consensus.cc:2804] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:08.865150  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.865175  4107 raft_consensus.cc:493] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.863181  3397 leader_election.cc:304] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.867287  4103 raft_consensus.cc:2749] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.866508  4107 raft_consensus.cc:3060] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:08.862167  4177 heartbeater.cc:499] Master 127.27.63.126:36891 was elected leader, sending a full tablet report...
W20260709 13:14:08.873127  3323 leader_election.cc:343] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.873539  3323 leader_election.cc:304] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.874826  4105 raft_consensus.cc:493] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.875505  4105 raft_consensus.cc:515] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.875950  3985 raft_consensus.cc:2749] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.832291  4106 raft_consensus.cc:515] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.893713  4105 leader_election.cc:290] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:08.895252  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:08.895933  3352 raft_consensus.cc:2468] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:08.897217  3396 leader_election.cc:304] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:08.898066  4105 raft_consensus.cc:2804] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:08.898485  4105 raft_consensus.cc:493] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.898928  4105 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:08.899319  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.902649  4085 raft_consensus.cc:493] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.903146  4085 raft_consensus.cc:515] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.903548  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
W20260709 13:14:08.904965  3471 leader_election.cc:343] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.905541  4085 leader_election.cc:290] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.907245  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:14:08.908478  3471 leader_election.cc:343] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.912096  4103 raft_consensus.cc:493] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.912640  4103 raft_consensus.cc:515] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.913390  4106 leader_election.cc:290] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.914264  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e493d066c44520bd444d8afc63ec91" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:08.914745  4103 leader_election.cc:290] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.915942  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:08.904928  4107 raft_consensus.cc:515] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.916469  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:14:08.916944  3397 leader_election.cc:343] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.914825  3430 raft_consensus.cc:3060] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:08.920372  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:08.920962  4150 raft_consensus.cc:3060] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:08.922345  4103 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.922910  4103 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.926824  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:08.928519  4150 raft_consensus.cc:2468] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:08.916469  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.918148  4107 leader_election.cc:290] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:08.918872  3470 leader_election.cc:304] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.931365  4106 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.931842  4106 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.919271  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:08.934787  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.938706  4107 raft_consensus.cc:493] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.939256  4107 raft_consensus.cc:515] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.940260  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
W20260709 13:14:08.941604  3471 leader_election.cc:343] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.941406  4107 leader_election.cc:290] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.946436  4079 raft_consensus.cc:2749] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.948833  4103 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:08.954870  3430 raft_consensus.cc:2468] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 2.
I20260709 13:14:08.957589  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:08.960525  3397 leader_election.cc:304] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:08.961493  4106 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.965301  4105 raft_consensus.cc:515] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.966722  4102 ts_tablet_manager.cc:1434] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.108s	user 0.045s	sys 0.052s
I20260709 13:14:08.967602  4102 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:08.968742  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:08.969576  3358 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:08.962087  3397 leader_election.cc:343] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.962903  3996 raft_consensus.cc:2804] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:08.970491  3397 leader_election.cc:304] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.970666  3996 raft_consensus.cc:697] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:08.962842  3985 raft_consensus.cc:493] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
W20260709 13:14:08.971838  3397 leader_election.cc:343] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.971853  3996 consensus_queue.cc:237] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.972586  3397 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.973593  4107 raft_consensus.cc:2749] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:08.974437  4107 raft_consensus.cc:2749] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:08.975734  3985 raft_consensus.cc:515] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.976940  3358 raft_consensus.cc:2468] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:08.977852  3985 leader_election.cc:290] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:08.978150  3396 leader_election.cc:304] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:08.979262  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:08.979312  4103 raft_consensus.cc:2804] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:08.979600  3470 leader_election.cc:304] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:08.980350  4103 raft_consensus.cc:697] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:08.979262  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.981797  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.981492  4103 consensus_queue.cc:237] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:08.982491  4150 raft_consensus.cc:2468] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:08.982748  4106 raft_consensus.cc:493] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.983786  4079 raft_consensus.cc:2804] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:08.984539  4079 raft_consensus.cc:697] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:08.985395  4079 consensus_queue.cc:237] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:08.983842  4106 raft_consensus.cc:515] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:08.989230  4106 leader_election.cc:290] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:08.989853  4105 leader_election.cc:290] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:08.990473  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:08.995754  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:08.993628  3323 leader_election.cc:304] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
W20260709 13:14:08.997344  3471 leader_election.cc:343] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.997772  4104 raft_consensus.cc:2804] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:08.998329  4104 raft_consensus.cc:493] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:08.998814  4104 raft_consensus.cc:3060] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.002163  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.011137  3470 leader_election.cc:304] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.053673  3985 raft_consensus.cc:493] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.019222  4085 raft_consensus.cc:493] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
W20260709 13:14:09.011868  3397 leader_election.cc:343] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.993877  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:09.035337  3667 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.060226  3397 leader_election.cc:304] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.050499  4189 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.012230  4027 raft_consensus.cc:2749] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:09.059273  3985 raft_consensus.cc:515] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.060717  3667 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.058313  3181 catalog_manager.cc:5697] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:09.059298  3178 catalog_manager.cc:5697] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 2 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.022396  4193 raft_consensus.cc:493] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
W20260709 13:14:09.061089  3397 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 49253.9us
I20260709 13:14:09.085451  4193 raft_consensus.cc:515] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.033586  4104 raft_consensus.cc:515] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.087574  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.081835  4197 raft_consensus.cc:493] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.088708  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:09.059345  4085 raft_consensus.cc:515] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.088439  4197 raft_consensus.cc:515] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.066841  3996 raft_consensus.cc:2749] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:09.089557  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:09.090971  4193 leader_election.cc:290] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:09.093091  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.080888  4102 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:09.089401  3427 raft_consensus.cc:2468] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:09.093937  4148 raft_consensus.cc:2393] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 2: Already voted for candidate 06964934536148c2aa77a2f1c6382fea in this term.
W20260709 13:14:09.092247  3323 leader_election.cc:343] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.094785  3667 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:09.095101  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.092420  3423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f058e8e6fdfa480db59f5004728b8f80" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:09.091414  3502 raft_consensus.cc:3060] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.091214  3985 leader_election.cc:290] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:09.094753  3322 leader_election.cc:304] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.092324  3351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:09.097025  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.097550  4085 leader_election.cc:290] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.088089  4104 leader_election.cc:290] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.102018  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.103154  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:09.095723  4150 raft_consensus.cc:3060] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.103813  3427 raft_consensus.cc:2468] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:09.097759  3351 raft_consensus.cc:2393] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 2: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
I20260709 13:14:09.052520  4196 raft_consensus.cc:493] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
W20260709 13:14:09.105105  3471 leader_election.cc:343] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.105325  4196 raft_consensus.cc:515] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.098315  3667 raft_consensus.cc:2804] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:14:09.098407  3323 leader_election.cc:343] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.088776  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d122fabfab34d9ea6e8bf2100aaf1e3" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.107137  3667 raft_consensus.cc:493] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.106794  3322 leader_election.cc:304] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.107657  3667 raft_consensus.cc:3060] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.084692  4189 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.099682  4102 tablet_bootstrap.cc:492] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:09.108711  4102 ts_tablet_manager.cc:1403] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.141s	user 0.041s	sys 0.037s
I20260709 13:14:09.109568  4196 leader_election.cc:290] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
W20260709 13:14:09.085704  3397 leader_election.cc:343] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.101174  4197 leader_election.cc:290] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:09.113339  4102 raft_consensus.cc:359] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.114040  4102 raft_consensus.cc:740] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:09.114792  4102 consensus_queue.cc:260] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.013429  4107 raft_consensus.cc:493] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.111271  3322 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.111656  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:09.118597  4102 ts_tablet_manager.cc:1434] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:14:09.111129  3985 raft_consensus.cc:3060] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.120419  3430 raft_consensus.cc:2468] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.102577  4085 raft_consensus.cc:493] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.120805  4102 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:09.122586  4197 raft_consensus.cc:2804] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:09.122828  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.123503  4148 raft_consensus.cc:2468] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.112844  4196 raft_consensus.cc:493] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.121584  4107 raft_consensus.cc:515] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.124280  4196 raft_consensus.cc:515] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.126820  4196 leader_election.cc:290] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:09.127105  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.134164  3471 leader_election.cc:304] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.134686  4148 raft_consensus.cc:2468] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.131749  4150 raft_consensus.cc:2468] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:09.131043  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:09.131597  4085 raft_consensus.cc:515] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.136101  3504 raft_consensus.cc:2468] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:09.112991  3996 raft_consensus.cc:493] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.122978  4197 raft_consensus.cc:493] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.140235  4197 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:09.140256  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:09.141676  3667 raft_consensus.cc:515] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.143535  3667 leader_election.cc:290] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:09.135334  3471 leader_election.cc:304] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:09.128376  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:09.127895  4107 leader_election.cc:290] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.150403  3667 raft_consensus.cc:493] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.145146  3398 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:09.150887  3667 raft_consensus.cc:515] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.145311  3471 leader_election.cc:304] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:09.147750  3323 leader_election.cc:304] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:09.151794  3351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:09.151965  4189 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.152520  3501 raft_consensus.cc:2468] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:09.152803  4200 raft_consensus.cc:2804] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.153080  4193 raft_consensus.cc:2804] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.148408  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.138839  3985 raft_consensus.cc:2749] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:09.151605  4105 raft_consensus.cc:2804] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.154572  3398 leader_election.cc:304] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:09.153453  4200 raft_consensus.cc:697] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:09.156704  3351 raft_consensus.cc:2468] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.150043  4027 raft_consensus.cc:2804] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.157382  4027 raft_consensus.cc:493] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.157734  4027 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.153551  4193 raft_consensus.cc:493] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.158241  4193 raft_consensus.cc:3060] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.158656  4200 consensus_queue.cc:237] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.164290  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4611ffabfbe4fa39f2fa67f79c3b048" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:09.164330  4105 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.164930  3426 raft_consensus.cc:3060] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.165225  4105 raft_consensus.cc:3060] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.131088  3502 raft_consensus.cc:2468] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:09.169839  4196 raft_consensus.cc:2749] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:09.175661  3423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:09.176365  3423 raft_consensus.cc:2468] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.176738  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.177620  3426 raft_consensus.cc:2468] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:09.177950  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.169186  4107 raft_consensus.cc:2804] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.178578  4107 raft_consensus.cc:493] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.178956  4107 raft_consensus.cc:3060] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.146245  3996 raft_consensus.cc:515] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.182016  3322 leader_election.cc:304] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:09.182726  3430 raft_consensus.cc:2468] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.182837  3985 raft_consensus.cc:2804] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.183424  3985 raft_consensus.cc:697] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
W20260709 13:14:09.189510  3323 leader_election.cc:343] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.188424  4103 raft_consensus.cc:493] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.191149  4103 raft_consensus.cc:515] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.191895  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:09.187891  3985 consensus_queue.cc:237] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.193290  4103 leader_election.cc:290] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.187968  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:09.195740  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.196269  3501 raft_consensus.cc:2468] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:09.196499  4150 raft_consensus.cc:2468] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:09.197600  3323 leader_election.cc:304] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:09.191162  3667 leader_election.cc:290] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.200314  4104 raft_consensus.cc:2804] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.207858  4079 raft_consensus.cc:493] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.208511  4079 raft_consensus.cc:515] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.209066  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.210687  4079 leader_election.cc:290] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:09.208055  4104 raft_consensus.cc:493] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.211470  4104 raft_consensus.cc:3060] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.201767  3398 leader_election.cc:304] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
W20260709 13:14:09.215480  3471 leader_election.cc:343] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.216923  4197 raft_consensus.cc:515] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.217582  4103 raft_consensus.cc:2804] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.218005  4103 raft_consensus.cc:493] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.218366  4103 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.219336  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.220022  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
W20260709 13:14:09.220980  3471 leader_election.cc:343] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.186767  4189 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.222018  4189 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.183435  3996 leader_election.cc:290] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.232118  4027 raft_consensus.cc:515] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.233620  4104 raft_consensus.cc:515] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.235101  4027 leader_election.cc:290] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:09.235718  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.220875  3501 raft_consensus.cc:2468] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:09.237018  3667 raft_consensus.cc:493] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.245584  3667 raft_consensus.cc:515] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.247079  4150 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.251151  4103 raft_consensus.cc:515] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.253722  4104 leader_election.cc:290] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.200122  4085 leader_election.cc:290] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.260944  3398 leader_election.cc:304] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
W20260709 13:14:09.263764  3470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:32907, user_credentials={real_user=slave}} blocked reactor thread for 83289.2us
I20260709 13:14:09.265645  3470 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.227890  4105 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.260414  4079 raft_consensus.cc:493] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.274170  3667 leader_election.cc:290] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
W20260709 13:14:09.275439  3324 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 55886.4us
I20260709 13:14:09.277035  4104 raft_consensus.cc:493] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.276930  4079 raft_consensus.cc:515] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.271636  4085 raft_consensus.cc:2804] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.273427  4197 leader_election.cc:290] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 3 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:09.275084  4189 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:09.284552  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:09.282600  4085 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.285578  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.292106  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.301317  4085 raft_consensus.cc:3060] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.312832  4079 leader_election.cc:290] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.313830  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.303359  4106 raft_consensus.cc:493] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.281124  4104 raft_consensus.cc:515] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.314620  4106 raft_consensus.cc:515] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:14:09.315419  3471 leader_election.cc:343] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.316007  4104 leader_election.cc:290] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.316812  4106 leader_election.cc:290] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:09.319382  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.304749  4107 raft_consensus.cc:515] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.308748  4149 raft_consensus.cc:3060] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.325551  4107 leader_election.cc:290] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.325903  4206 raft_consensus.cc:493] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.306107  4103 leader_election.cc:290] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
W20260709 13:14:09.340351  3397 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 73755.7us
W20260709 13:14:09.324100  3471 leader_election.cc:343] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.289588  4203 raft_consensus.cc:2804] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.326475  4206 raft_consensus.cc:515] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.363600  4206 leader_election.cc:290] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.364770  3499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:09.309252  3500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:09.365744  3501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f" is_pre_election: true
I20260709 13:14:09.366250  3500 raft_consensus.cc:2468] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:09.366461  3501 raft_consensus.cc:2468] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:09.365780  3499 raft_consensus.cc:2393] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 2: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:09.351462  4203 raft_consensus.cc:493] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.368422  3324 leader_election.cc:304] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:09.368971  4203 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.369424  4104 raft_consensus.cc:2804] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.369536  3324 leader_election.cc:304] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:09.369822  4104 raft_consensus.cc:493] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.370884  3667 raft_consensus.cc:2804] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:14:09.346761  3397 leader_election.cc:343] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.371385  3667 raft_consensus.cc:493] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.371830  3667 raft_consensus.cc:3060] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:09.372722  3397 leader_election.cc:343] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.370237  4104 raft_consensus.cc:3060] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.376304  3470 leader_election.cc:304] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.309268  4105 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.308954  4150 raft_consensus.cc:2468] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 2.
W20260709 13:14:09.379299  3398 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.68:33961, user_credentials={real_user=slave}} blocked reactor thread for 118452us
I20260709 13:14:09.380483  3470 leader_election.cc:304] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
W20260709 13:14:09.380533  3323 leader_election.cc:343] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.311439  4193 raft_consensus.cc:515] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.388413  3667 raft_consensus.cc:515] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.389876  3351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:09.391363  3351 raft_consensus.cc:2468] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 1.
I20260709 13:14:09.391445  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.391955  3499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:09.392400  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.393070  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.390806  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2" is_pre_election: true
I20260709 13:14:09.394176  3358 raft_consensus.cc:2468] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.383219  4195 raft_consensus.cc:2804] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.396260  3505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:09.396493  4145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.396744  4195 raft_consensus.cc:493] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.396955  3505 raft_consensus.cc:3060] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.397256  4195 raft_consensus.cc:3060] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.397641  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.398437  4151 raft_consensus.cc:2393] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 2: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:09.393095  4147 raft_consensus.cc:2468] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 1.
I20260709 13:14:09.400331  3500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:09.400925  3500 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.404839  3471 leader_election.cc:304] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:09.407595  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1537c0aaa1d470e930915f178ec10d2" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:09.408193  3430 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:09.411897  3500 raft_consensus.cc:2468] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:09.412287  4104 raft_consensus.cc:515] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.414574  4104 leader_election.cc:290] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.414733  3667 leader_election.cc:290] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.415069  3430 raft_consensus.cc:2468] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 3.
I20260709 13:14:09.421200  4106 raft_consensus.cc:2764] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:14:09.417214  4203 raft_consensus.cc:515] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.430124  3398 leader_election.cc:304] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:09.431107  3322 leader_election.cc:304] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.430125  3396 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:09.432667  3985 raft_consensus.cc:2804] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:09.433180  3985 raft_consensus.cc:697] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:09.435571  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:09.436566  3358 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.435567  3985 consensus_queue.cc:237] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.433677  4027 raft_consensus.cc:2804] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.436452  4189 raft_consensus.cc:2804] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.440557  4105 raft_consensus.cc:2804] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.454435  3500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
W20260709 13:14:09.457363  3323 leader_election.cc:343] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.441095  3505 raft_consensus.cc:2468] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:09.461684  3323 leader_election.cc:304] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.445334  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:09.453490  3504 raft_consensus.cc:2393] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83ef2126cf6d4f628cd668d0817871f2 in current term 2: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:09.448328  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:09.457275  3358 raft_consensus.cc:2468] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 2.
I20260709 13:14:09.460914  4105 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.462994  4189 raft_consensus.cc:697] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:09.471444  3500 raft_consensus.cc:3060] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.474570  4211 raft_consensus.cc:2749] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:09.480661  3502 raft_consensus.cc:3060] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.486306  4105 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.491618  3358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:09.508828  3358 raft_consensus.cc:2393] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 2: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
I20260709 13:14:09.510005  4149 raft_consensus.cc:2468] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:09.510911  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:09.511761  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.511752  3430 raft_consensus.cc:2468] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.512648  3323 leader_election.cc:304] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 73143d6f16c94a3aaef99f8e07588c1f
I20260709 13:14:09.513978  3667 raft_consensus.cc:2804] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.514436  3667 raft_consensus.cc:697] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:09.490402  4193 leader_election.cc:290] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:09.515297  3667 consensus_queue.cc:237] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.516651  3351 raft_consensus.cc:1275] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:09.519649  4203 leader_election.cc:290] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.519476  4212 consensus_queue.cc:1048] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:09.520310  4189 consensus_queue.cc:237] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.530481  4027 raft_consensus.cc:697] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:09.531617  4027 consensus_queue.cc:237] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.492282  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.536126  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:09.536820  3426 raft_consensus.cc:2468] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:09.539300  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bb4036c344c3cbf83650eeaf091aa" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.541033  3470 leader_election.cc:304] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.542850  4102 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:09.543741  4106 raft_consensus.cc:2804] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.544164  4106 raft_consensus.cc:493] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.544513  4106 raft_consensus.cc:3060] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.548931  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecac352d2640492ba17f310cbb649d7c" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.551178  4195 raft_consensus.cc:515] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
W20260709 13:14:09.561630  3323 leader_election.cc:343] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.572268  3500 raft_consensus.cc:2468] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:09.575353  4102 tablet_bootstrap.cc:492] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:09.576169  4102 ts_tablet_manager.cc:1403] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.456s	user 0.033s	sys 0.021s
W20260709 13:14:09.581112  3397 leader_election.cc:343] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:09.582042  3397 leader_election.cc:343] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.582707  3324 leader_election.cc:304] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
W20260709 13:14:09.582852  3397 leader_election.cc:343] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.583660  3398 leader_election.cc:304] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
W20260709 13:14:09.584573  3397 leader_election.cc:343] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.584236  4102 raft_consensus.cc:359] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.585584  4102 raft_consensus.cc:740] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
W20260709 13:14:09.585752  3397 leader_election.cc:343] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.567503  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5557da3aee9b41f793515bea5f1f42cc" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.587661  3667 raft_consensus.cc:2804] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.587884  4102 consensus_queue.cc:260] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.590710  3667 raft_consensus.cc:697] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:09.586954  3996 raft_consensus.cc:2804] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.602792  4102 ts_tablet_manager.cc:1434] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
W20260709 13:14:09.603031  3471 consensus_peers.cc:597] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:09.604772  3996 raft_consensus.cc:697] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:09.606038  3470 leader_election.cc:304] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.606168  3996 consensus_queue.cc:237] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.607538  4196 raft_consensus.cc:2804] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.608119  4196 raft_consensus.cc:493] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.608562  4196 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.613842  4195 leader_election.cc:290] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.614311  4105 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.623688  4105 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:09.596536  3504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:09.624859  3352 raft_consensus.cc:1275] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:09.625252  3504 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.607941  3470 leader_election.cc:304] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.607589  3502 raft_consensus.cc:2468] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:09.590797  4085 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.631449  4085 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.596604  3667 consensus_queue.cc:237] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.633152  3499 raft_consensus.cc:2393] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 06964934536148c2aa77a2f1c6382fea in current term 2: Already voted for candidate 73143d6f16c94a3aaef99f8e07588c1f in this term.
I20260709 13:14:09.633252  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.597096  4149 raft_consensus.cc:3060] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:09.640275  3471 leader_election.cc:343] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.642693  3431 raft_consensus.cc:1275] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:09.644757  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7574b0cc43f4903a703314b82b0e9da" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:09.645304  3431 raft_consensus.cc:3060] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.645413  3324 leader_election.cc:304] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:09.646838  3667 raft_consensus.cc:2804] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.634260  4223 raft_consensus.cc:2804] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:09.650560  3667 raft_consensus.cc:697] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:09.652547  3667 consensus_queue.cc:237] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.653707  3430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:09.654479  3430 raft_consensus.cc:2393] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 2: Already voted for candidate 06964934536148c2aa77a2f1c6382fea in this term.
I20260709 13:14:09.655289  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.656201  4223 raft_consensus.cc:493] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:09.656646  4223 raft_consensus.cc:3060] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:09.656996  3471 leader_election.cc:343] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.661126  3431 raft_consensus.cc:2468] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 2.
I20260709 13:14:09.640954  4105 consensus_queue.cc:1048] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:14:09.663178  3398 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea; no voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282
W20260709 13:14:09.682011  3397 leader_election.cc:343] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.673251  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:09.684589  3470 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.673573  4102 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:09.673507  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.674237  4146 raft_consensus.cc:1275] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:09.673334  4144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.675550  4079 consensus_queue.cc:1048] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:09.679184  3504 raft_consensus.cc:2468] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:09.679157  4106 raft_consensus.cc:515] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:09.697968  3398 leader_election.cc:304] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
W20260709 13:14:09.688863  3397 consensus_peers.cc:597] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:09.689793  3470 leader_election.cc:304] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.701860  4226 raft_consensus.cc:2804] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.704849  4226 raft_consensus.cc:697] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:09.705536  4106 leader_election.cc:290] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:09.705694  4226 consensus_queue.cc:237] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.684521  4105 raft_consensus.cc:2749] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:14:09.709815  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.698546  3352 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.703590  4079 raft_consensus.cc:2804] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.714725  4079 raft_consensus.cc:697] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:09.711797  3354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40971e1ab1f04b21a166d7eac0ee7d95" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:09.716656  3354 raft_consensus.cc:3060] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.715662  4079 consensus_queue.cc:237] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:09.692319  4195 raft_consensus.cc:2749] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:14:09.722162  3471 leader_election.cc:343] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.735690  3352 raft_consensus.cc:2468] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:09.736486  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a5322a6cb9422f85206f14c490fb6a" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.739537  4148 raft_consensus.cc:2393] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 73143d6f16c94a3aaef99f8e07588c1f in current term 2: Already voted for candidate 83ef2126cf6d4f628cd668d0817871f2 in this term.
I20260709 13:14:09.740806  3471 leader_election.cc:304] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:09.743986  4195 raft_consensus.cc:2749] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:14:09.731356  3180 catalog_manager.cc:5697] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 2 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.737166  3396 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:09.748822  3179 catalog_manager.cc:5697] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 2 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:09.759475  4196 raft_consensus.cc:515] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
W20260709 13:14:09.764601  3323 consensus_peers.cc:597] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:09.763725  4149 raft_consensus.cc:2468] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:09.731504  4223 raft_consensus.cc:515] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:09.769094  4105 raft_consensus.cc:2804] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.775337  4105 raft_consensus.cc:697] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
W20260709 13:14:09.775820  3397 leader_election.cc:343] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.775693  4203 consensus_queue.cc:1048] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:09.776237  4105 consensus_queue.cc:237] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
W20260709 13:14:09.776870  3397 leader_election.cc:343] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.787725  4146 raft_consensus.cc:1275] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:09.789598  3667 consensus_queue.cc:1048] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:09.789849  3504 raft_consensus.cc:1275] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:09.791566  3179 catalog_manager.cc:5697] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.794466  3179 catalog_manager.cc:5697] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:09.800577  3179 catalog_manager.cc:5697] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.803373  3179 catalog_manager.cc:5697] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.807578  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:09.808938  3667 consensus_queue.cc:1048] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:09.810484  3471 leader_election.cc:343] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:09.818521  4223 leader_election.cc:290] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:09.860870  3431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:09.861577  3431 raft_consensus.cc:3060] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:09.820775  4239 log.cc:927] Time spent T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:14:09.810235  4196 leader_election.cc:290] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:09.882047  3431 raft_consensus.cc:2468] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 2.
I20260709 13:14:09.863096  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:09.889827  3470 leader_election.cc:304] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:09.931635  4102 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:09.932466  3423 raft_consensus.cc:1275] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:09.895051  3427 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.896792  4106 raft_consensus.cc:2804] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.954898  3180 catalog_manager.cc:5697] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 2 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.969280  4102 tablet_bootstrap.cc:492] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:09.970016  4102 ts_tablet_manager.cc:1403] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.297s	user 0.030s	sys 0.021s
I20260709 13:14:09.984459  4144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ff273fcfdc45a386458728eb7254f3" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
W20260709 13:14:09.988600  4236 log.cc:927] Time spent T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:14:09.991521  3354 raft_consensus.cc:2468] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 2.
I20260709 13:14:09.973627  4106 raft_consensus.cc:697] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:10.007650  4106 consensus_queue.cc:237] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.010730  4102 raft_consensus.cc:359] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.027204  3427 raft_consensus.cc:2468] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 2.
I20260709 13:14:10.028337  4102 raft_consensus.cc:740] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.034547  4102 consensus_queue.cc:260] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.030236  3470 leader_election.cc:304] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:10.036441  3426 raft_consensus.cc:1275] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:10.038303  4102 ts_tablet_manager.cc:1434] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.001s
I20260709 13:14:10.040454  4085 raft_consensus.cc:2804] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:10.043712  4102 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.050323  3985 consensus_queue.cc:1048] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:14:10.047135  4241 log.cc:927] Time spent T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:14:10.058739  3470 leader_election.cc:304] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 972a4469a4654416ab0a153d8c740282
I20260709 13:14:10.060207  3499 raft_consensus.cc:1275] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.069741  4202 raft_consensus.cc:2804] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:10.070323  4202 raft_consensus.cc:697] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
W20260709 13:14:10.078052  3471 leader_election.cc:343] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:10.077729  3996 consensus_queue.cc:1048] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.082839  3667 consensus_queue.cc:1048] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:10.086323  4242 log.cc:927] Time spent T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f: Append to log took a long time: real 0.134s	user 0.004s	sys 0.000s
I20260709 13:14:10.088241  4144 raft_consensus.cc:1275] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.089761  4207 consensus_queue.cc:1048] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.090778  3505 raft_consensus.cc:1275] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.094204  3985 consensus_queue.cc:1048] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.071173  4202 consensus_queue.cc:237] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.115433  3178 catalog_manager.cc:5697] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 2 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:10.125958  3323 consensus_peers.cc:597] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:10.123410  3178 catalog_manager.cc:5697] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 2 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:10.115377  4085 raft_consensus.cc:697] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:10.154536  3500 raft_consensus.cc:1275] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.157802  3500 raft_consensus.cc:1275] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.157401  4085 consensus_queue.cc:237] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:10.167021  3667 consensus_queue.cc:1048] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:10.174472  3426 raft_consensus.cc:1275] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.167004  4226 consensus_queue.cc:1048] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:14:10.180388  4228 consensus_queue.cc:1048] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.183195  4144 raft_consensus.cc:1275] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:14:10.199713  3323 consensus_peers.cc:597] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:10.203974  3178 catalog_manager.cc:5697] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 2 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:10.207783  3178 catalog_manager.cc:5697] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 2 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:10.225235  3397 consensus_peers.cc:597] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:10.249287  3397 consensus_peers.cc:597] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:10.250528  3397 consensus_peers.cc:597] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:10.264055  3178 catalog_manager.cc:5697] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 2 to 3, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 3 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:10.273007  3471 consensus_peers.cc:597] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:10.273470  4147 raft_consensus.cc:1275] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.296720  4249 consensus_queue.cc:1048] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:10.300592  3499 raft_consensus.cc:1275] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.303053  3996 consensus_queue.cc:1048] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.304978  4102 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.306100  4102 tablet_bootstrap.cc:492] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.306728  4102 ts_tablet_manager.cc:1403] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.263s	user 0.019s	sys 0.000s
W20260709 13:14:10.306864  3397 consensus_peers.cc:597] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:10.305002  3499 raft_consensus.cc:1275] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.309984  4209 consensus_queue.cc:1048] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.315579  4102 raft_consensus.cc:359] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.317219  4102 raft_consensus.cc:740] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.317970  4209 consensus_queue.cc:1048] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.328888  4102 consensus_queue.cc:260] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.338953  3354 raft_consensus.cc:1275] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.344709  4226 consensus_queue.cc:1048] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:14:10.351110  3397 consensus_peers.cc:597] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:10.351789  4102 ts_tablet_manager.cc:1434] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:14:10.352965  4102 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
W20260709 13:14:10.359937  3397 consensus_peers.cc:597] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:10.369436  3354 raft_consensus.cc:1275] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.372614  3179 catalog_manager.cc:5697] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 2 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:10.374732  3179 catalog_manager.cc:5697] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 2 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:10.377168  3179 catalog_manager.cc:5697] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 2 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:10.385387  3397 consensus_peers.cc:597] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:10.427325  4207 consensus_queue.cc:1048] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
W20260709 13:14:10.448580  3471 consensus_peers.cc:597] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:10.449172  3323 consensus_peers.cc:597] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:10.463505  3427 raft_consensus.cc:1275] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.465246  4218 consensus_queue.cc:1048] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:10.468895  3323 consensus_peers.cc:597] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:10.473002  3323 consensus_peers.cc:597] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:10.487027  3178 catalog_manager.cc:5697] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:10.503381  3397 consensus_peers.cc:597] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:10.519721  3471 consensus_peers.cc:597] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:10.532490  4102 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.533700  4102 tablet_bootstrap.cc:492] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.534397  4102 ts_tablet_manager.cc:1403] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.182s	user 0.018s	sys 0.008s
I20260709 13:14:10.536965  4102 raft_consensus.cc:359] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.537771  4102 raft_consensus.cc:740] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.538441  4102 consensus_queue.cc:260] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.540442  4102 ts_tablet_manager.cc:1434] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:10.541190  4102 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.563390  4150 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:10.567034  4102 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.568060  4102 tablet_bootstrap.cc:492] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.568719  4102 ts_tablet_manager.cc:1403] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 13:14:10.570734  3427 raft_consensus.cc:1275] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.571327  4102 raft_consensus.cc:359] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.572008  4102 raft_consensus.cc:740] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.572579  4207 consensus_queue.cc:1048] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.572887  4102 consensus_queue.cc:260] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.574651  4102 ts_tablet_manager.cc:1434] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:10.575420  4102 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.577598  4150 raft_consensus.cc:1275] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.579119  4207 consensus_queue.cc:1048] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.596098  4102 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.597024  4102 tablet_bootstrap.cc:492] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.597640  4102 ts_tablet_manager.cc:1403] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 13:14:10.600060  4102 raft_consensus.cc:359] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.600718  4102 raft_consensus.cc:740] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.601348  4102 consensus_queue.cc:260] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.602811  4102 ts_tablet_manager.cc:1434] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:10.603575  4102 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.620955  4102 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.621897  4102 tablet_bootstrap.cc:492] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.622511  4102 ts_tablet_manager.cc:1403] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.000s
I20260709 13:14:10.624897  4102 raft_consensus.cc:359] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.625567  4102 raft_consensus.cc:740] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.626241  4102 consensus_queue.cc:260] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.628297  4102 ts_tablet_manager.cc:1434] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
W20260709 13:14:10.628683  3323 consensus_peers.cc:597] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:10.629048  4102 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.671211  4102 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.671975  4102 tablet_bootstrap.cc:492] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.672547  4102 ts_tablet_manager.cc:1403] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.044s	user 0.026s	sys 0.012s
I20260709 13:14:10.674639  4102 raft_consensus.cc:359] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.675143  4102 raft_consensus.cc:740] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.675710  4102 consensus_queue.cc:260] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.677482  4102 ts_tablet_manager.cc:1434] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:10.678469  4102 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.691602  4102 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.692433  4102 tablet_bootstrap.cc:492] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.692993  4102 ts_tablet_manager.cc:1403] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 13:14:10.694641  3349 raft_consensus.cc:1275] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.695158  4102 raft_consensus.cc:359] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.695837  4102 raft_consensus.cc:740] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.695984  4106 consensus_queue.cc:1048] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.696511  4102 consensus_queue.cc:260] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.699004  4102 ts_tablet_manager.cc:1434] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:14:10.700282  4102 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.714944  4102 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.715890  4102 tablet_bootstrap.cc:492] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.716544  4102 ts_tablet_manager.cc:1403] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.003s
I20260709 13:14:10.718894  4102 raft_consensus.cc:359] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.719457  4102 raft_consensus.cc:740] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
W20260709 13:14:10.719633  3471 consensus_peers.cc:597] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:10.720124  4102 consensus_queue.cc:260] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.725788  4102 ts_tablet_manager.cc:1434] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:14:10.726747  4102 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.745826  4102 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.746732  4102 tablet_bootstrap.cc:492] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.747489  4102 ts_tablet_manager.cc:1403] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.001s
I20260709 13:14:10.749364  4102 raft_consensus.cc:359] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:10.749800  4102 raft_consensus.cc:740] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.750247  4102 consensus_queue.cc:260] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:10.751842  4102 ts_tablet_manager.cc:1434] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.000s
I20260709 13:14:10.752884  4102 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.766497  4102 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.767467  4102 tablet_bootstrap.cc:492] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.768242  4102 ts_tablet_manager.cc:1403] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.003s
I20260709 13:14:10.770668  4102 raft_consensus.cc:359] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.771171  4102 raft_consensus.cc:740] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.771739  4102 consensus_queue.cc:260] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.773244  4102 ts_tablet_manager.cc:1434] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:10.774130  4102 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.787464  4102 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.788332  4102 tablet_bootstrap.cc:492] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.788867  4102 ts_tablet_manager.cc:1403] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:14:10.790594  4102 raft_consensus.cc:359] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.791126  4102 raft_consensus.cc:740] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.791718  4102 consensus_queue.cc:260] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.793612  4102 ts_tablet_manager.cc:1434] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:10.794461  4102 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
W20260709 13:14:10.794493  3397 consensus_peers.cc:597] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:10.809397  4102 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.810226  4102 tablet_bootstrap.cc:492] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.810843  4102 ts_tablet_manager.cc:1403] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:14:10.812774  4102 raft_consensus.cc:359] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:10.813299  4102 raft_consensus.cc:740] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.813848  4102 consensus_queue.cc:260] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:10.815589  4102 ts_tablet_manager.cc:1434] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:10.816591  4102 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.837214  4102 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.838053  4102 tablet_bootstrap.cc:492] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.838627  4102 ts_tablet_manager.cc:1403] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 13:14:10.840583  4102 raft_consensus.cc:359] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.841162  4102 raft_consensus.cc:740] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.841763  4102 consensus_queue.cc:260] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.843832  4102 ts_tablet_manager.cc:1434] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:10.844928  4102 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.869964  4102 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.870895  4102 tablet_bootstrap.cc:492] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.871515  4102 ts_tablet_manager.cc:1403] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.012s
I20260709 13:14:10.874074  4102 raft_consensus.cc:359] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.874718  4102 raft_consensus.cc:740] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.875352  4102 consensus_queue.cc:260] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.877337  4102 ts_tablet_manager.cc:1434] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:14:10.878628  4102 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.891065  4102 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.891887  4102 tablet_bootstrap.cc:492] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.892556  4102 ts_tablet_manager.cc:1403] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:14:10.895793  4102 raft_consensus.cc:359] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.896788  4102 raft_consensus.cc:740] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.897401  4102 consensus_queue.cc:260] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.899400  4102 ts_tablet_manager.cc:1434] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:14:10.900198  4102 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.912560  4102 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.913342  4102 tablet_bootstrap.cc:492] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.913888  4102 ts_tablet_manager.cc:1403] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:14:10.915705  4102 raft_consensus.cc:359] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.916345  4102 raft_consensus.cc:740] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.916926  4102 consensus_queue.cc:260] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.918678  4102 ts_tablet_manager.cc:1434] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:10.920189  4102 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.943531  4102 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.944406  4102 tablet_bootstrap.cc:492] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.944994  4102 ts_tablet_manager.cc:1403] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.007s
I20260709 13:14:10.946630  4102 raft_consensus.cc:359] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:10.947120  4102 raft_consensus.cc:740] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.947638  4102 consensus_queue.cc:260] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:10.949195  4102 ts_tablet_manager.cc:1434] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:10.950237  4102 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.963896  4102 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.964787  4102 tablet_bootstrap.cc:492] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.965569  4102 ts_tablet_manager.cc:1403] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.001s
I20260709 13:14:10.967860  4102 raft_consensus.cc:359] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.968515  4102 raft_consensus.cc:740] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.969174  4102 consensus_queue.cc:260] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:10.971293  4102 ts_tablet_manager.cc:1434] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:14:10.972034  4102 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:10.985430  4102 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:10.986116  4102 tablet_bootstrap.cc:492] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:10.986631  4102 ts_tablet_manager.cc:1403] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:14:10.988526  4102 raft_consensus.cc:359] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.989013  4102 raft_consensus.cc:740] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:10.989522  4102 consensus_queue.cc:260] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:10.991420  4102 ts_tablet_manager.cc:1434] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:10.992660  4102 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.023089  4102 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.023977  4102 tablet_bootstrap.cc:492] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.024785  4102 ts_tablet_manager.cc:1403] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.007s
I20260709 13:14:11.028849  4102 raft_consensus.cc:359] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.029516  4102 raft_consensus.cc:740] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.030205  4102 consensus_queue.cc:260] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.032332  4102 ts_tablet_manager.cc:1434] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:11.034231  4102 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.048025  4102 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.048956  4102 tablet_bootstrap.cc:492] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.049561  4102 ts_tablet_manager.cc:1403] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.016s	user 0.004s	sys 0.010s
I20260709 13:14:11.051349  4102 raft_consensus.cc:359] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.051805  4102 raft_consensus.cc:740] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.052297  4102 consensus_queue.cc:260] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.053784  4102 ts_tablet_manager.cc:1434] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:14:11.055392  4102 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.071985  4102 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.072880  4102 tablet_bootstrap.cc:492] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.073537  4102 ts_tablet_manager.cc:1403] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 13:14:11.075956  4150 raft_consensus.cc:3060] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.076030  4102 raft_consensus.cc:359] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:11.076930  4102 raft_consensus.cc:740] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.077612  4102 consensus_queue.cc:260] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:11.079639  4102 ts_tablet_manager.cc:1434] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:14:11.081465  4102 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.104269  4102 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.105142  4102 tablet_bootstrap.cc:492] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.105742  4102 ts_tablet_manager.cc:1403] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.024s	user 0.022s	sys 0.001s
I20260709 13:14:11.108245  4102 raft_consensus.cc:359] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.108716  4102 raft_consensus.cc:740] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.109346  4102 consensus_queue.cc:260] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.111792  4102 ts_tablet_manager.cc:1434] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:14:11.112556  4102 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.130450  4102 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.131376  4102 tablet_bootstrap.cc:492] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.132125  4102 ts_tablet_manager.cc:1403] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:14:11.134524  4102 raft_consensus.cc:359] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:11.135167  4102 raft_consensus.cc:740] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.135787  4102 consensus_queue.cc:260] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:11.137502  4102 ts_tablet_manager.cc:1434] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:14:11.138227  4102 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.153218  4102 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.154136  4102 tablet_bootstrap.cc:492] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.154784  4102 ts_tablet_manager.cc:1403] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:14:11.157150  4102 raft_consensus.cc:359] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.157786  4102 raft_consensus.cc:740] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.158485  4102 consensus_queue.cc:260] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.160485  4102 ts_tablet_manager.cc:1434] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:14:11.162292  4102 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.182889  4102 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.183823  4102 tablet_bootstrap.cc:492] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.184491  4102 ts_tablet_manager.cc:1403] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 13:14:11.186782  4102 raft_consensus.cc:359] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.187438  4102 raft_consensus.cc:740] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.188119  4102 consensus_queue.cc:260] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:11.190181  4102 ts_tablet_manager.cc:1434] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:11.192366  4102 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.201988  4150 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.227234  4147 raft_consensus.cc:3060] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.237258  4102 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.238581  4102 tablet_bootstrap.cc:492] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.239423  4102 ts_tablet_manager.cc:1403] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.023s
I20260709 13:14:11.242674  4102 raft_consensus.cc:359] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:11.243409  4102 raft_consensus.cc:740] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.244208  4102 consensus_queue.cc:260] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:11.246757  4102 ts_tablet_manager.cc:1434] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:11.247855  4102 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Bootstrap starting.
I20260709 13:14:11.255791  4146 raft_consensus.cc:3060] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.269253  4149 raft_consensus.cc:3060] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.276803  4150 raft_consensus.cc:3060] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.319950  4147 raft_consensus.cc:3060] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.329846  4102 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:11.330859  4102 tablet_bootstrap.cc:492] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Bootstrap complete.
I20260709 13:14:11.331549  4102 ts_tablet_manager.cc:1403] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Time spent bootstrapping tablet: real 0.084s	user 0.019s	sys 0.035s
I20260709 13:14:11.338433  4102 raft_consensus.cc:359] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:11.338997  4146 raft_consensus.cc:3060] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.339257  4102 raft_consensus.cc:740] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Initialized, Role: FOLLOWER
I20260709 13:14:11.341378  4102 consensus_queue.cc:260] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:11.345088  4102 ts_tablet_manager.cc:1434] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.001s
I20260709 13:14:11.420588  4150 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.505867  4146 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.561403  4218 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.561923  4218 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:11.565003  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea" is_pre_election: true
I20260709 13:14:11.565018  4218 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:11.565331  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282" is_pre_election: true
I20260709 13:14:11.565729  3427 raft_consensus.cc:2468] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 2.
I20260709 13:14:11.566020  4146 raft_consensus.cc:2468] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 1.
I20260709 13:14:11.566803  3470 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:11.567577  4218 raft_consensus.cc:2804] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:11.567935  4218 raft_consensus.cc:493] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.568293  4218 raft_consensus.cc:3060] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:11.574079  4218 raft_consensus.cc:515] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:11.575491  4218 leader_election.cc:290] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 3 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
I20260709 13:14:11.576241  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:11.576467  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9ccb9fa24984b6c9a11b8429cd45338" candidate_uuid: "73143d6f16c94a3aaef99f8e07588c1f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "972a4469a4654416ab0a153d8c740282"
I20260709 13:14:11.576812  3427 raft_consensus.cc:3060] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:11.577126  4146 raft_consensus.cc:3060] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:14:11.582588  3427 raft_consensus.cc:2468] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 3.
I20260709 13:14:11.582811  4146 raft_consensus.cc:2468] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73143d6f16c94a3aaef99f8e07588c1f in term 3.
I20260709 13:14:11.583709  3470 leader_election.cc:304] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:11.584407  4207 raft_consensus.cc:2804] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:11.584820  4207 raft_consensus.cc:697] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 LEADER]: Becoming Leader. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:11.585573  4207 consensus_queue.cc:237] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:11.594054  3179 catalog_manager.cc:5697] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f reported cstate change: term changed from 1 to 3, leader changed from 972a4469a4654416ab0a153d8c740282 (127.27.63.65) to 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68). New cstate: current_term: 3 leader_uuid: "73143d6f16c94a3aaef99f8e07588c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:11.721688  4149 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.749547  4147 raft_consensus.cc:3060] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:14:11.864447  3397 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:34091, user_credentials={real_user=slave}} blocked reactor thread for 54458.1us
I20260709 13:14:11.988636  3426 raft_consensus.cc:1275] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:11.990422  4218 consensus_queue.cc:1048] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:12.020700  4147 raft_consensus.cc:1275] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Refusing update from remote peer 73143d6f16c94a3aaef99f8e07588c1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:12.022042  4218 consensus_queue.cc:1048] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:18.690284 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:18.751655  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:18.752417  3426 raft_consensus.cc:606] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:18.755157  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:18.755604  3426 raft_consensus.cc:606] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:18.766223  3349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1"
dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
mode: GRACEFUL
new_leader_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51424
I20260709 13:14:18.766839  3349 raft_consensus.cc:606] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Received request to transfer leadership to TS 06964934536148c2aa77a2f1c6382fea
I20260709 13:14:18.769372  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "691ec311298b4d19a1e62763ea313587"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "83ef2126cf6d4f628cd668d0817871f2"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:18.769767  3426 raft_consensus.cc:606] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Received request to transfer leadership to TS 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:18.780957  3502 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b"
dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56324
I20260709 13:14:18.781490  3502 raft_consensus.cc:606] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:18.784030  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:18.784508  3426 raft_consensus.cc:606] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:18.785597 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:18.786793 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:18.886317  4348 raft_consensus.cc:993] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 83ef2126cf6d4f628cd668d0817871f2 to start an election
I20260709 13:14:18.886992  4343 raft_consensus.cc:1081] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Signalling peer 83ef2126cf6d4f628cd668d0817871f2 to start an election
I20260709 13:14:18.888375  3349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "691ec311298b4d19a1e62763ea313587"
dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
 from {username='slave'} at 127.0.0.1:33750
I20260709 13:14:18.888898  3349 raft_consensus.cc:493] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:18.889189  3349 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:18.895723  3349 raft_consensus.cc:515] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:18.898381  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:18.898712  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:18.899091  3426 raft_consensus.cc:3055] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:18.899271  3502 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:18.899444  3426 raft_consensus.cc:740] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:18.900429  3426 consensus_queue.cc:260] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:18.901440  3426 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:18.897804  3349 leader_election.cc:290] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:18.906550  3502 raft_consensus.cc:2468] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:18.908478  3324 leader_election.cc:304] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:18.908792  3426 raft_consensus.cc:2468] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 2.
I20260709 13:14:18.909260  4322 raft_consensus.cc:2804] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:18.920554  4322 raft_consensus.cc:697] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:18.921317  4322 consensus_queue.cc:237] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:18.939719  3179 catalog_manager.cc:5697] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 1 to 2, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 2 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:18.969112  4343 raft_consensus.cc:993] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:18.969548  4348 raft_consensus.cc:1081] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:18.970878  4146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:18.971331  4146 raft_consensus.cc:493] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:18.971688  4146 raft_consensus.cc:3060] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:18.978338  4146 raft_consensus.cc:515] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:18.980711  4146 leader_election.cc:290] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:18.998250  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:18.998862  3426 raft_consensus.cc:3055] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:18.999161  3426 raft_consensus.cc:740] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:18.999908  3426 consensus_queue.cc:260] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.000782  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e38e5f24604ea48097687a200d7e48" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:19.000988  3426 raft_consensus.cc:3060] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.001384  3502 raft_consensus.cc:3060] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.006785  3426 raft_consensus.cc:2468] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:19.007539  3502 raft_consensus.cc:2468] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:19.008531  4097 leader_election.cc:304] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:19.010584  4362 raft_consensus.cc:2804] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.012064  4362 raft_consensus.cc:697] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:19.014047  4362 consensus_queue.cc:237] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.026850  3179 catalog_manager.cc:5697] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 1 to 2, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.153493  4343 raft_consensus.cc:993] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:19.153996  4348 raft_consensus.cc:1081] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:19.155548  4146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:19.156111  4146 raft_consensus.cc:493] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.156484  4146 raft_consensus.cc:3060] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.156564  4079 raft_consensus.cc:993] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:19.157207  4349 raft_consensus.cc:1081] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:19.158743  4150 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58278
I20260709 13:14:19.159235  4150 raft_consensus.cc:493] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.159509  4150 raft_consensus.cc:3060] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.162737  4146 raft_consensus.cc:515] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.164561  4146 leader_election.cc:290] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:19.165184  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:19.165597  3500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd7e8a9e3e248c4ba469d477a27f8cb" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:19.165813  3426 raft_consensus.cc:3055] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:19.166132  4150 raft_consensus.cc:515] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.166170  3500 raft_consensus.cc:3060] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.166145  3426 raft_consensus.cc:740] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:19.167428  3426 consensus_queue.cc:260] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.167948  4150 leader_election.cc:290] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:19.168735  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:19.168731  3426 raft_consensus.cc:3060] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.168977  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ac2072e92b45fd89cb3b4057fb9e7b" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:19.169420  3427 raft_consensus.cc:3060] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.169585  3502 raft_consensus.cc:3055] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:19.169996  3502 raft_consensus.cc:740] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:19.170724  3502 consensus_queue.cc:260] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.171828  3502 raft_consensus.cc:3060] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.174978  3500 raft_consensus.cc:2468] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:19.175613  3426 raft_consensus.cc:2468] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:19.176304  3427 raft_consensus.cc:2468] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:19.176891  4099 leader_election.cc:304] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:19.177819  4362 raft_consensus.cc:2804] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.178129  4097 leader_election.cc:304] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:19.178318  4362 raft_consensus.cc:697] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:19.179625  3502 raft_consensus.cc:2468] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:19.179294  4362 consensus_queue.cc:237] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.180397  4363 raft_consensus.cc:2804] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.181764  4363 raft_consensus.cc:697] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:19.182606  4363 consensus_queue.cc:237] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.196902  3179 catalog_manager.cc:5697] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 1 to 2, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.198545  3179 catalog_manager.cc:5697] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 1 to 2, leader changed from 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.226727  4343 raft_consensus.cc:993] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:19.227144  4348 raft_consensus.cc:1081] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:19.228520  4146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:19.228986  4146 raft_consensus.cc:493] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.229353  4146 raft_consensus.cc:3060] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.234548  4146 raft_consensus.cc:515] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:19.236241  4146 leader_election.cc:290] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:19.237720  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:19.238301  3427 raft_consensus.cc:3055] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:19.238579  3427 raft_consensus.cc:740] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:19.239281  3427 consensus_queue.cc:260] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:19.240305  3427 raft_consensus.cc:3060] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.245131  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22507d010a9a43f6834e5a2dfd53c148" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:19.245728  3352 raft_consensus.cc:3060] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.246594  3427 raft_consensus.cc:2468] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:19.248250  4097 leader_election.cc:304] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:19.249116  4363 raft_consensus.cc:2804] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.249550  4363 raft_consensus.cc:697] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:19.250325  4363 consensus_queue.cc:237] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:19.252398  3352 raft_consensus.cc:2468] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:19.274606  3179 catalog_manager.cc:5697] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 1 to 2, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.287086  4354 raft_consensus.cc:993] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2: : Instructing follower 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:19.287622  4354 raft_consensus.cc:1081] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Signalling peer 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:19.289319  3427 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51740
I20260709 13:14:19.289819  3427 raft_consensus.cc:493] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.290158  3427 raft_consensus.cc:3060] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.295174  3427 raft_consensus.cc:515] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:19.297470  3427 leader_election.cc:290] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:19.298030  3349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:19.298594  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65d7199324524f1fbb9a40b53925a3a1" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:19.298615  3349 raft_consensus.cc:3055] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:19.299157  3349 raft_consensus.cc:740] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:19.299206  3502 raft_consensus.cc:3060] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.300032  3349 consensus_queue.cc:260] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:19.301240  3349 raft_consensus.cc:3060] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.305686  3502 raft_consensus.cc:2468] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:19.306874  3398 leader_election.cc:304] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:19.307612  3349 raft_consensus.cc:2468] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:19.307720  4343 raft_consensus.cc:2804] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.308269  4343 raft_consensus.cc:697] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:19.309113  4343 consensus_queue.cc:237] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:19.318745  3179 catalog_manager.cc:5697] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.439203  3426 raft_consensus.cc:1275] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.440747  4354 consensus_queue.cc:1048] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.445756  3502 raft_consensus.cc:1275] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.446995  4363 consensus_queue.cc:1048] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.451018  3502 raft_consensus.cc:1275] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.452386  4354 consensus_queue.cc:1048] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.508616  3426 raft_consensus.cc:1275] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.510389  4362 consensus_queue.cc:1048] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.627243  3503 raft_consensus.cc:1275] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.628727  4365 consensus_queue.cc:1048] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.638166  3426 raft_consensus.cc:1275] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.639643  4363 consensus_queue.cc:1048] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.695457  3426 raft_consensus.cc:1275] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.696841  4363 consensus_queue.cc:1048] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.706919  3503 raft_consensus.cc:1275] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.708899  4363 consensus_queue.cc:1048] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.781456  3349 raft_consensus.cc:1275] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.782526  4348 consensus_queue.cc:1048] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.788911 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:19.803114  3503 raft_consensus.cc:1275] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.812935  4378 consensus_queue.cc:1048] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:19.828352  3352 raft_consensus.cc:1275] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.829763  4363 consensus_queue.cc:1048] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.851459  3427 raft_consensus.cc:1275] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:19.853919  4365 consensus_queue.cc:1048] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:19.865160 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:19.916669  3503 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03a915f11e804f04adb461457918ee75"
dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
mode: GRACEFUL
new_leader_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:56324
I20260709 13:14:19.917264  3503 raft_consensus.cc:606] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Received request to transfer leadership to TS 06964934536148c2aa77a2f1c6382fea
I20260709 13:14:19.920118  3352 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "691ec311298b4d19a1e62763ea313587"
dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
mode: GRACEFUL
new_leader_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51424
I20260709 13:14:19.926630  3352 raft_consensus.cc:606] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Received request to transfer leadership to TS 06964934536148c2aa77a2f1c6382fea
I20260709 13:14:19.929301  3503 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1128833b31748d0b098825fd4212f82"
dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56324
I20260709 13:14:19.929821  3503 raft_consensus.cc:606] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:19.932250  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:19.932801  3426 raft_consensus.cc:606] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:19.935022  3503 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83"
dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
mode: GRACEFUL
new_leader_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:56324
I20260709 13:14:19.935498  3503 raft_consensus.cc:606] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Received request to transfer leadership to TS 06964934536148c2aa77a2f1c6382fea
I20260709 13:14:19.937526  3503 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a"
dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56324
I20260709 13:14:19.937968  3503 raft_consensus.cc:606] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:19.938835 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:19.940028 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:19.944548  4369 raft_consensus.cc:993] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f: : Instructing follower 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:19.945025  4349 raft_consensus.cc:1081] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Signalling peer 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:19.946489  3426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03a915f11e804f04adb461457918ee75"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51730
I20260709 13:14:19.947024  3426 raft_consensus.cc:493] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.947479  3426 raft_consensus.cc:3060] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.950865  4343 raft_consensus.cc:993] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:19.951401  4348 raft_consensus.cc:1081] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:19.952890  4147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:19.953315  4147 raft_consensus.cc:493] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.953648  4147 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:19.954583  3426 raft_consensus.cc:515] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:19.956028  3426 leader_election.cc:290] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:19.956745  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:19.956928  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a915f11e804f04adb461457918ee75" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:19.957304  3352 raft_consensus.cc:3060] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.957630  3503 raft_consensus.cc:3055] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:19.957954  3503 raft_consensus.cc:740] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:19.958784  3503 consensus_queue.cc:260] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:19.960791  3503 raft_consensus.cc:3060] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.963557  3352 raft_consensus.cc:2468] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:19.964694  3396 leader_election.cc:304] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:19.965507  4348 raft_consensus.cc:2804] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.965963  4348 raft_consensus.cc:697] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:19.966719  4348 consensus_queue.cc:237] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:19.967990  4147 raft_consensus.cc:515] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.970651  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:19.970970  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d414c916bc43f2a6579cf5fa1beaee" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:19.971287  3426 raft_consensus.cc:3055] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:19.971602  3502 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:19.971817  3426 raft_consensus.cc:740] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:19.972702  3426 consensus_queue.cc:260] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.973654  3426 raft_consensus.cc:3060] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:19.976513  3503 raft_consensus.cc:2468] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:19.980178  4147 leader_election.cc:290] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:19.980939  3502 raft_consensus.cc:2468] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:19.981891  4099 leader_election.cc:304] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:19.982879  4383 raft_consensus.cc:2804] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:19.983314  4383 raft_consensus.cc:697] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:19.984122  4383 consensus_queue.cc:237] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:19.983858  3426 raft_consensus.cc:2468] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:19.988422  3179 catalog_manager.cc:5697] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.998346  3180 catalog_manager.cc:5697] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 2 to 3, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 3 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:20.010504  4322 raft_consensus.cc:993] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2: : Instructing follower 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:20.011055  4344 raft_consensus.cc:1081] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Signalling peer 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:20.012394  3426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "691ec311298b4d19a1e62763ea313587"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51740
I20260709 13:14:20.012836  3426 raft_consensus.cc:493] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.013128  3426 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:20.017668  3426 raft_consensus.cc:515] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:20.019117  3426 leader_election.cc:290] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 3 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:20.020006  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:20.020147  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691ec311298b4d19a1e62763ea313587" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:20.020623  3352 raft_consensus.cc:3055] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:20.020802  3502 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:20.020942  3352 raft_consensus.cc:740] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:20.021688  3352 consensus_queue.cc:260] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:20.022575  3352 raft_consensus.cc:3060] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:20.025689  3502 raft_consensus.cc:2468] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 3.
I20260709 13:14:20.026708  3398 leader_election.cc:304] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:20.027451  4378 raft_consensus.cc:2804] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:20.027449  3352 raft_consensus.cc:2468] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 3.
I20260709 13:14:20.028014  4378 raft_consensus.cc:697] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 3 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:20.028929  4378 consensus_queue.cc:237] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:20.037334  3180 catalog_manager.cc:5697] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 2 to 3, leader changed from 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 3 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.280745  4394 raft_consensus.cc:993] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f: : Instructing follower 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:20.281270  4394 raft_consensus.cc:1081] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Signalling peer 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:20.282828  3426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51730
I20260709 13:14:20.283521  3426 raft_consensus.cc:493] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.283946  3426 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.285590  4369 raft_consensus.cc:993] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:20.286139  4349 raft_consensus.cc:1081] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:20.287710  4147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58278
I20260709 13:14:20.288208  4147 raft_consensus.cc:493] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.288475  4147 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:20.292814  3426 raft_consensus.cc:515] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:20.295183  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:20.295689  3352 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.295650  4147 raft_consensus.cc:515] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:20.296669  3502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:20.297204  3502 raft_consensus.cc:3055] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:20.297470  3502 raft_consensus.cc:740] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:20.298146  3502 consensus_queue.cc:260] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:20.298513  4147 leader_election.cc:290] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:20.298993  3349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:20.299170  3502 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.299543  3349 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:20.300969  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e30e8276a41a98a3e4f6cf043375a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:20.301522  3503 raft_consensus.cc:3055] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:20.301859  3503 raft_consensus.cc:740] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:20.301613  3352 raft_consensus.cc:2468] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:20.302608  3503 consensus_queue.cc:260] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:20.303071  3396 leader_election.cc:304] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:20.303791  3503 raft_consensus.cc:3060] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:20.303900  4343 raft_consensus.cc:2804] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:20.304431  4343 raft_consensus.cc:697] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:20.305155  4343 consensus_queue.cc:237] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:20.306052  3349 raft_consensus.cc:2468] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:20.307036  4097 leader_election.cc:304] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:20.307901  4383 raft_consensus.cc:2804] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:20.308375  4383 raft_consensus.cc:697] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:20.309209  4383 consensus_queue.cc:237] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:20.314671  3503 raft_consensus.cc:2468] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:20.316200  3426 leader_election.cc:290] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:20.316591  3502 raft_consensus.cc:2468] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:20.325939  3180 catalog_manager.cc:5697] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 2 to 3, leader changed from 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 3 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.328718  3179 catalog_manager.cc:5697] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.358312  3502 raft_consensus.cc:1275] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.359911  4378 consensus_queue.cc:1048] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.377882  3349 raft_consensus.cc:1275] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.379316  4378 consensus_queue.cc:1048] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.419029  4369 raft_consensus.cc:993] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:20.419500  4369 raft_consensus.cc:1081] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:20.420975  4150 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1128833b31748d0b098825fd4212f82"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58278
I20260709 13:14:20.421482  4150 raft_consensus.cc:493] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.421763  4150 raft_consensus.cc:3060] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.426800  4150 raft_consensus.cc:515] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:20.429219  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:20.429298  4150 leader_election.cc:290] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:20.429478  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1128833b31748d0b098825fd4212f82" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:20.429808  3426 raft_consensus.cc:3060] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.430106  3503 raft_consensus.cc:3055] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:20.430424  3503 raft_consensus.cc:740] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:20.431136  3503 consensus_queue.cc:260] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:20.432466  3503 raft_consensus.cc:3060] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.435895  3426 raft_consensus.cc:2468] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:20.437031  4097 leader_election.cc:304] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:20.437249  3503 raft_consensus.cc:2468] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:20.437834  4383 raft_consensus.cc:2804] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:20.438382  4383 raft_consensus.cc:697] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:20.439139  4383 consensus_queue.cc:237] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:20.448622  3180 catalog_manager.cc:5697] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 1 to 2, leader changed from 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:20.486860  3503 raft_consensus.cc:1275] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:20.487088  3349 raft_consensus.cc:1275] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:20.488351  4343 consensus_queue.cc:1048] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:20.488260  4383 consensus_queue.cc:1048] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:20.492228  3426 raft_consensus.cc:1275] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:20.494561  4383 consensus_queue.cc:1048] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:20.500526  3502 raft_consensus.cc:1275] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:20.503587  4378 consensus_queue.cc:1048] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:20.773968  3349 raft_consensus.cc:1275] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.775168  4343 consensus_queue.cc:1048] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.784442  3503 raft_consensus.cc:1275] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.786653  4378 consensus_queue.cc:1048] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.792232  3503 raft_consensus.cc:1275] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:20.793722  4383 consensus_queue.cc:1048] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:20.814231  3349 raft_consensus.cc:1275] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:20.815809  4363 consensus_queue.cc:1048] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:20.867285  3426 raft_consensus.cc:1275] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.868541  4363 consensus_queue.cc:1048] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.893028  3502 raft_consensus.cc:1275] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.894439  4383 consensus_queue.cc:1048] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.940902 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:20.983393 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:21.033491  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:21.033996  3426 raft_consensus.cc:606] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:21.036469  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:21.037003  3426 raft_consensus.cc:606] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:21.039234  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:21.039678  3426 raft_consensus.cc:606] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:21.041836  3502 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e"
dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
mode: GRACEFUL
new_leader_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:56324
I20260709 13:14:21.042342  3502 raft_consensus.cc:606] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Received request to transfer leadership to TS 06964934536148c2aa77a2f1c6382fea
I20260709 13:14:21.044646  3502 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50"
dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
mode: GRACEFUL
new_leader_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:56324
I20260709 13:14:21.045120  3502 raft_consensus.cc:606] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Received request to transfer leadership to TS 06964934536148c2aa77a2f1c6382fea
I20260709 13:14:21.047488  3352 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7e3b18f25574f46ac56a124bb52c1c3"
dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:51424
I20260709 13:14:21.048096  3352 raft_consensus.cc:606] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:21.049297 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:21.050827 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:21.232666  4349 raft_consensus.cc:993] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f: : Instructing follower 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:21.233204  4395 raft_consensus.cc:1081] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Signalling peer 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:21.234489  3427 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51730
I20260709 13:14:21.234967  3427 raft_consensus.cc:493] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:21.235244  3427 raft_consensus.cc:3060] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.240147  3427 raft_consensus.cc:515] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:21.241918  3427 leader_election.cc:290] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:21.243069  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:21.243230  3349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f23621b920b44c8dbdd783bed09dfa50" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:21.243685  3503 raft_consensus.cc:3055] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:21.243779  3349 raft_consensus.cc:3060] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.244073  3503 raft_consensus.cc:740] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:21.244874  3503 consensus_queue.cc:260] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:21.246044  3503 raft_consensus.cc:3060] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.249722  3349 raft_consensus.cc:2468] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:21.250739  3396 leader_election.cc:304] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:21.251420  4348 raft_consensus.cc:2804] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:21.251345  3503 raft_consensus.cc:2468] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:21.252101  4348 raft_consensus.cc:697] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:21.252987  4348 consensus_queue.cc:237] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:21.260946  3181 catalog_manager.cc:5697] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.307698  4348 raft_consensus.cc:993] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:21.308171  4378 raft_consensus.cc:1081] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:21.309476  4147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:21.310036  4147 raft_consensus.cc:493] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:21.310299  4147 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:21.316609  4147 raft_consensus.cc:515] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:21.318825  4147 leader_election.cc:290] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:21.319024  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:21.319640  3426 raft_consensus.cc:3055] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:21.319954  3426 raft_consensus.cc:740] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:21.320631  3426 consensus_queue.cc:260] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:21.321473  3426 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:21.324419  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042f5a22d47425eac26b6f03be99d9d" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:21.324978  3503 raft_consensus.cc:3060] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:21.325891  3426 raft_consensus.cc:2468] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:21.328344  4097 leader_election.cc:304] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:21.329234  4383 raft_consensus.cc:2804] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:21.329775  4383 raft_consensus.cc:697] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:21.330579  4383 consensus_queue.cc:237] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:21.332305  3503 raft_consensus.cc:2468] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:21.344610  3181 catalog_manager.cc:5697] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 2 to 3, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 3 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:21.394008  4344 raft_consensus.cc:993] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:21.394544  4344 raft_consensus.cc:1081] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:21.395912  4147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7e3b18f25574f46ac56a124bb52c1c3"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58292
I20260709 13:14:21.396435  4147 raft_consensus.cc:493] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:21.396803  4147 raft_consensus.cc:3060] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.401845  4147 raft_consensus.cc:515] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:21.403115  4147 leader_election.cc:290] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961), 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623)
I20260709 13:14:21.403816  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e3b18f25574f46ac56a124bb52c1c3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:21.403983  3349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e3b18f25574f46ac56a124bb52c1c3" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:21.404448  3503 raft_consensus.cc:3060] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.404520  3349 raft_consensus.cc:3055] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:21.404891  3349 raft_consensus.cc:740] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:21.405658  3349 consensus_queue.cc:260] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:21.406805  3349 raft_consensus.cc:3060] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.410806  3503 raft_consensus.cc:2468] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:21.411947  4099 leader_election.cc:304] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:21.412763  4383 raft_consensus.cc:2804] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:21.413239  4383 raft_consensus.cc:697] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:21.413252  3349 raft_consensus.cc:2468] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:21.414050  4383 consensus_queue.cc:237] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } }
I20260709 13:14:21.424510  4348 raft_consensus.cc:993] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:21.425028  4348 raft_consensus.cc:1081] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:21.426493  4150 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:21.427045  4150 raft_consensus.cc:493] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:21.427416  4150 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:21.437249  4150 raft_consensus.cc:515] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:21.439581  4150 leader_election.cc:290] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:21.441392  3349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:21.441952  3349 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:21.443132  3181 catalog_manager.cc:5697] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 1 to 2, leader changed from 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.446669  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2d92deaf5c4766a2cc54da4c274434" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:21.447302  3427 raft_consensus.cc:3055] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:21.447633  3427 raft_consensus.cc:740] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:21.448326  3427 consensus_queue.cc:260] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:21.449481  3427 raft_consensus.cc:3060] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:21.449883  4349 raft_consensus.cc:993] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f: : Instructing follower 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:21.450364  4349 raft_consensus.cc:1081] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Signalling peer 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:21.451035  3349 raft_consensus.cc:2468] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:21.452032  4097 leader_election.cc:304] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:21.453073  4365 raft_consensus.cc:2804] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:21.453540  4365 raft_consensus.cc:697] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:21.454351  4365 consensus_queue.cc:237] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:21.454913  3423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51730
I20260709 13:14:21.455523  3423 raft_consensus.cc:493] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:21.455839  3423 raft_consensus.cc:3060] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.457741  3427 raft_consensus.cc:2468] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:21.462553  3423 raft_consensus.cc:515] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:21.465449  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:21.466022  3503 raft_consensus.cc:3055] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:21.466298  3503 raft_consensus.cc:740] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:21.467062  3503 consensus_queue.cc:260] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:21.467868  3503 raft_consensus.cc:3060] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.475646  3503 raft_consensus.cc:2468] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:21.476198  3423 leader_election.cc:290] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:21.476528  3398 leader_election.cc:304] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 73143d6f16c94a3aaef99f8e07588c1f; no voters: 
I20260709 13:14:21.477355  4378 raft_consensus.cc:2804] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:21.477568  3349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1bb6275bf5a44debe9b7cdd0d757a3e" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:21.478793  4378 raft_consensus.cc:697] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:21.478904  3349 raft_consensus.cc:3060] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.479740  4378 consensus_queue.cc:237] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:21.476756  3181 catalog_manager.cc:5697] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 2 to 3, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 3 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.485656  3349 raft_consensus.cc:2468] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:21.493467  3178 catalog_manager.cc:5697] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.610759  4343 raft_consensus.cc:993] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:21.611238  4343 raft_consensus.cc:1081] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:21.612890  4150 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:21.613600  4150 raft_consensus.cc:493] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:21.614040  4150 raft_consensus.cc:3060] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.619421  4150 raft_consensus.cc:515] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:21.621230  4150 leader_election.cc:290] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:21.621932  3349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:21.622221  3423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208caff3bfdf46a495a86f06a667659a" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:21.622615  3349 raft_consensus.cc:3060] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.622856  3423 raft_consensus.cc:3055] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:21.623241  3423 raft_consensus.cc:740] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:21.624022  3423 consensus_queue.cc:260] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:21.625264  3423 raft_consensus.cc:3060] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:21.627605  3349 raft_consensus.cc:2468] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:21.628741  4097 leader_election.cc:304] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:21.629518  4365 raft_consensus.cc:2804] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:21.629976  4365 raft_consensus.cc:697] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:21.630779  4365 consensus_queue.cc:237] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:21.631587  3423 raft_consensus.cc:2468] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:21.640527  3178 catalog_manager.cc:5697] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 1 to 2, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.669112  3349 raft_consensus.cc:1275] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:21.670214  4378 consensus_queue.cc:1048] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:21.680009  3503 raft_consensus.cc:1275] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:21.687824  4378 consensus_queue.cc:1048] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:21.802240  3502 raft_consensus.cc:1275] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:21.806120  4422 consensus_queue.cc:1048] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:21.864431  3423 raft_consensus.cc:1275] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:21.866362  4422 consensus_queue.cc:1048] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:21.919696  3349 raft_consensus.cc:1275] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:21.921540  4363 consensus_queue.cc:1048] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:21.942955  3426 raft_consensus.cc:1275] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:21.948916  4422 consensus_queue.cc:1048] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:21.992702  3354 raft_consensus.cc:1275] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:21.994793  4363 consensus_queue.cc:1048] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:22.025112  3426 raft_consensus.cc:1275] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:22.026995  4363 consensus_queue.cc:1048] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:22.034878  3503 raft_consensus.cc:1275] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:22.036721  4365 consensus_queue.cc:1048] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:22.053328 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:22.061820  3352 raft_consensus.cc:1275] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:22.075328  4422 consensus_queue.cc:1048] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:22.099623  3354 raft_consensus.cc:1275] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:22.103715  4348 consensus_queue.cc:1048] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:22.106714  3500 raft_consensus.cc:1275] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:22.112568  4441 consensus_queue.cc:1048] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:22.190552 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:22.231134  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:22.231742  3426 raft_consensus.cc:606] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:22.234109  3500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417"
dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56324
I20260709 13:14:22.234524  3500 raft_consensus.cc:606] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:22.237098  3426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "83ef2126cf6d4f628cd668d0817871f2"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:22.237685  3426 raft_consensus.cc:606] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Received request to transfer leadership to TS 83ef2126cf6d4f628cd668d0817871f2
I20260709 13:14:22.238889 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:22.240495 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:22.273682  4395 raft_consensus.cc:993] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:22.274360  4395 raft_consensus.cc:1081] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:22.276376  4147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58278
I20260709 13:14:22.277503  4147 raft_consensus.cc:493] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:22.277897  4147 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:22.287513  4147 raft_consensus.cc:515] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:22.289230  4147 leader_election.cc:290] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:22.289974  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:22.290233  3500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1d609ecf684c38b48d776083ce2417" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:22.290839  3500 raft_consensus.cc:3055] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:22.290647  3426 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:22.291188  3500 raft_consensus.cc:740] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 73143d6f16c94a3aaef99f8e07588c1f, State: Running, Role: LEADER
I20260709 13:14:22.292192  3500 consensus_queue.cc:260] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:22.293502  3500 raft_consensus.cc:3060] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:22.297392  4348 raft_consensus.cc:993] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 83ef2126cf6d4f628cd668d0817871f2 to start an election
I20260709 13:14:22.298071  4449 raft_consensus.cc:1081] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Signalling peer 83ef2126cf6d4f628cd668d0817871f2 to start an election
I20260709 13:14:22.299333  3426 raft_consensus.cc:2468] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:22.299877  3352 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83"
dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
 from {username='slave'} at 127.0.0.1:33750
I20260709 13:14:22.300518  3352 raft_consensus.cc:493] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:22.300850  3352 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:22.301334  4097 leader_election.cc:304] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:22.301620  3500 raft_consensus.cc:2468] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:22.302495  4383 raft_consensus.cc:2804] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:22.303000  4383 raft_consensus.cc:697] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:22.303999  4383 consensus_queue.cc:237] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:22.308490  3352 raft_consensus.cc:515] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:22.310298  3352 leader_election.cc:290] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 3 election: Requested vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:22.311187  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:22.311786  3426 raft_consensus.cc:3055] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:22.312088  3426 raft_consensus.cc:740] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:22.312180  3500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad53e24ea0f24ba6977b81fbb4608a83" candidate_uuid: "83ef2126cf6d4f628cd668d0817871f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:22.312708  3500 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:22.312970  3426 consensus_queue.cc:260] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:22.314301  3426 raft_consensus.cc:3060] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:22.319849  3500 raft_consensus.cc:2468] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 3.
I20260709 13:14:22.321201  3324 leader_election.cc:304] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:22.322254  4322 raft_consensus.cc:2804] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:22.322665  4322 raft_consensus.cc:697] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 LEADER]: Becoming Leader. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:22.323395  4322 consensus_queue.cc:237] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:22.334995  3426 raft_consensus.cc:2468] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef2126cf6d4f628cd668d0817871f2 in term 3.
I20260709 13:14:22.335111  3178 catalog_manager.cc:5697] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 2 to 3, leader changed from 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 3 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:22.345747  3180 catalog_manager.cc:5697] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 reported cstate change: term changed from 2 to 3, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66). New cstate: current_term: 3 leader_uuid: "83ef2126cf6d4f628cd668d0817871f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.532336  4410 raft_consensus.cc:993] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:22.532774  4449 raft_consensus.cc:1081] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:22.534119  4147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:22.534597  4147 raft_consensus.cc:493] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:22.534916  4147 raft_consensus.cc:3060] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:22.539924  4147 raft_consensus.cc:515] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:22.541481  4147 leader_election.cc:290] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:22.542434  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:22.542434  3426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ae6143fac5245deabc5979617a4f018" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:22.543229  3352 raft_consensus.cc:3060] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:22.543529  3426 raft_consensus.cc:3055] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:22.543844  3426 raft_consensus.cc:740] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:22.544669  3426 consensus_queue.cc:260] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:22.545854  3426 raft_consensus.cc:3060] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:22.549845  3352 raft_consensus.cc:2468] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:22.550895  4097 leader_election.cc:304] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:22.551883  4445 raft_consensus.cc:2804] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:22.552371  4445 raft_consensus.cc:697] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:22.552697  3426 raft_consensus.cc:2468] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 2.
I20260709 13:14:22.553284  4445 consensus_queue.cc:237] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:22.564756  3180 catalog_manager.cc:5697] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 1 to 2, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.722656  3423 raft_consensus.cc:1275] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:22.723889  4344 consensus_queue.cc:1048] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:22.734417  3503 raft_consensus.cc:1275] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Refusing update from remote peer 83ef2126cf6d4f628cd668d0817871f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:22.736249  4434 consensus_queue.cc:1048] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:22.800752  3503 raft_consensus.cc:1275] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:22.802124  4363 consensus_queue.cc:1048] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:22.804447  3423 raft_consensus.cc:1275] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:22.805747  4363 consensus_queue.cc:1048] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:22.949057  3352 raft_consensus.cc:1275] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:22.950359  4445 consensus_queue.cc:1048] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:22.964340  3423 raft_consensus.cc:1275] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:22.965559  4363 consensus_queue.cc:1048] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:23.241425 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:23.285418 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:23.325454  3354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2"
dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
mode: GRACEFUL
new_leader_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51424
I20260709 13:14:23.326139  3354 raft_consensus.cc:606] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Received request to transfer leadership to TS 06964934536148c2aa77a2f1c6382fea
I20260709 13:14:23.327334 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:14:23.328554 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:23.383737  4322 raft_consensus.cc:993] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2: : Instructing follower 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:23.384291  4344 raft_consensus.cc:1081] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Signalling peer 06964934536148c2aa77a2f1c6382fea to start an election
I20260709 13:14:23.385660  3423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
 from {username='slave'} at 127.0.0.1:51740
I20260709 13:14:23.386193  3423 raft_consensus.cc:493] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:23.386590  3423 raft_consensus.cc:3060] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:23.396196  3423 raft_consensus.cc:515] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:23.398046  3423 leader_election.cc:290] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 73143d6f16c94a3aaef99f8e07588c1f (127.27.63.68:33961)
I20260709 13:14:23.398648  3352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:23.399258  3352 raft_consensus.cc:3055] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:23.399611  3352 raft_consensus.cc:740] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83ef2126cf6d4f628cd668d0817871f2, State: Running, Role: LEADER
I20260709 13:14:23.400415  3352 consensus_queue.cc:260] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:23.401613  3352 raft_consensus.cc:3060] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:23.404620  3503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8051a5cc00534c788d643e0b21d3d9c2" candidate_uuid: "06964934536148c2aa77a2f1c6382fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73143d6f16c94a3aaef99f8e07588c1f"
I20260709 13:14:23.405140  3503 raft_consensus.cc:3060] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:23.407877  3352 raft_consensus.cc:2468] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:23.409139  3396 leader_election.cc:304] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06964934536148c2aa77a2f1c6382fea, 83ef2126cf6d4f628cd668d0817871f2; no voters: 
I20260709 13:14:23.409744  4410 raft_consensus.cc:2804] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:23.410115  4410 raft_consensus.cc:697] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Leader. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:23.410832  4410 consensus_queue.cc:237] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } }
I20260709 13:14:23.411497  3503 raft_consensus.cc:2468] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06964934536148c2aa77a2f1c6382fea in term 2.
I20260709 13:14:23.426119  3180 catalog_manager.cc:5697] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea reported cstate change: term changed from 1 to 2, leader changed from 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66) to 06964934536148c2aa77a2f1c6382fea (127.27.63.67). New cstate: current_term: 2 leader_uuid: "06964934536148c2aa77a2f1c6382fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:23.848795  3354 raft_consensus.cc:1275] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:23.849959  4449 consensus_queue.cc:1048] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:23.863313  3500 raft_consensus.cc:1275] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Refusing update from remote peer 06964934536148c2aa77a2f1c6382fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:23.864524  4449 consensus_queue.cc:1048] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:24.329326 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:24.385005 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:24.419788  3427 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0"
dest_uuid: "06964934536148c2aa77a2f1c6382fea"
mode: GRACEFUL
new_leader_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:14:24.420459  3427 raft_consensus.cc:606] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Received request to transfer leadership to TS 972a4469a4654416ab0a153d8c740282
I20260709 13:14:24.421607 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:14:24.422684 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:24.522194  4475 raft_consensus.cc:993] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea: : Instructing follower 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:24.522720  4449 raft_consensus.cc:1081] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Signalling peer 972a4469a4654416ab0a153d8c740282 to start an election
I20260709 13:14:24.524308  4147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0"
dest_uuid: "972a4469a4654416ab0a153d8c740282"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:14:24.524835  4147 raft_consensus.cc:493] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:24.525146  4147 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.533628  4147 raft_consensus.cc:515] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:24.535023  4147 leader_election.cc:290] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Requested vote from peers 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623), 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907)
I20260709 13:14:24.535730  3349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "83ef2126cf6d4f628cd668d0817871f2"
I20260709 13:14:24.536154  3427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ff345dd8d4467829eded70f16f6a0" candidate_uuid: "972a4469a4654416ab0a153d8c740282" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06964934536148c2aa77a2f1c6382fea"
I20260709 13:14:24.536381  3349 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.536772  3427 raft_consensus.cc:3055] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:24.537062  3427 raft_consensus.cc:740] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06964934536148c2aa77a2f1c6382fea, State: Running, Role: LEADER
I20260709 13:14:24.537732  3427 consensus_queue.cc:260] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:24.538573  3427 raft_consensus.cc:3060] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.542644  3349 raft_consensus.cc:2468] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:24.543694  4097 leader_election.cc:304] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef2126cf6d4f628cd668d0817871f2, 972a4469a4654416ab0a153d8c740282; no voters: 
I20260709 13:14:24.544471  4383 raft_consensus.cc:2804] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:24.544936  4383 raft_consensus.cc:697] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Becoming Leader. State: Replica: 972a4469a4654416ab0a153d8c740282, State: Running, Role: LEADER
I20260709 13:14:24.545820  4383 consensus_queue.cc:237] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } }
I20260709 13:14:24.547498  3427 raft_consensus.cc:2468] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972a4469a4654416ab0a153d8c740282 in term 3.
I20260709 13:14:24.554386  3180 catalog_manager.cc:5697] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 reported cstate change: term changed from 2 to 3, leader changed from 06964934536148c2aa77a2f1c6382fea (127.27.63.67) to 972a4469a4654416ab0a153d8c740282 (127.27.63.65). New cstate: current_term: 3 leader_uuid: "972a4469a4654416ab0a153d8c740282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:25.108822  3352 raft_consensus.cc:1275] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:25.110014  4383 consensus_queue.cc:1048] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef2126cf6d4f628cd668d0817871f2" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:25.121969  3427 raft_consensus.cc:1275] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Refusing update from remote peer 972a4469a4654416ab0a153d8c740282: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:25.123370  4363 consensus_queue.cc:1048] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:25.423476 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:25.461129 27901 tablet_server.cc:179] TabletServer@127.27.63.65:34091 shutting down...
W20260709 13:14:25.488451  3323 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 53 similar messages]
W20260709 13:14:25.494346  3323 consensus_peers.cc:597] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.501669 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:25.502722 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.503863 27901 raft_consensus.cc:2243] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.504933 27901 raft_consensus.cc:2272] T a9d414c916bc43f2a6579cf5fa1beaee P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.507339 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.507977 27901 raft_consensus.cc:2243] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.508426 27901 raft_consensus.cc:2272] T 87e493d066c44520bd444d8afc63ec91 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.510475 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.510998 27901 raft_consensus.cc:2243] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.511327 27901 raft_consensus.cc:2272] T 9f6db7a8007940d6b8bc088af66589ae P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.513304 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.513883 27901 raft_consensus.cc:2243] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.514559 27901 raft_consensus.cc:2272] T 208caff3bfdf46a495a86f06a667659a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.517048 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.517670 27901 raft_consensus.cc:2243] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.518182 27901 raft_consensus.cc:2272] T d1ff273fcfdc45a386458728eb7254f3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.520481 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.521023 27901 raft_consensus.cc:2243] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.521781 27901 raft_consensus.cc:2272] T 9042f5a22d47425eac26b6f03be99d9d P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.524422 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.525036 27901 raft_consensus.cc:2243] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.525413 27901 raft_consensus.cc:2272] T 59159d79fbfc4835a64e52a2783f46f7 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.527238 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.527817 27901 raft_consensus.cc:2243] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.528285 27901 raft_consensus.cc:2272] T bb18f85836cc456b8362bb08424827ca P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.530193 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.530714 27901 raft_consensus.cc:2243] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.531395 27901 raft_consensus.cc:2272] T b61ff345dd8d4467829eded70f16f6a0 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.534296 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.534886 27901 raft_consensus.cc:2243] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.535261 27901 raft_consensus.cc:2272] T 361bb4036c344c3cbf83650eeaf091aa P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.537286 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.537971 27901 raft_consensus.cc:2243] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.538424 27901 raft_consensus.cc:2272] T f4611ffabfbe4fa39f2fa67f79c3b048 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.540675 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.541234 27901 raft_consensus.cc:2243] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.541579 27901 raft_consensus.cc:2272] T 094d0615636b4da4a2c8ba92b4218624 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.543568 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.544086 27901 raft_consensus.cc:2243] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.544508 27901 raft_consensus.cc:2272] T 3061a90b8dbc42488cec94032788e597 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.546500 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.547035 27901 raft_consensus.cc:2243] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.547729 27901 raft_consensus.cc:2272] T bf2d92deaf5c4766a2cc54da4c274434 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.550122 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.550853 27901 raft_consensus.cc:2243] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.551417 27901 raft_consensus.cc:2272] T f2794d9308d0458aa11c33cb7dab1748 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.553779 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.554380 27901 raft_consensus.cc:2243] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.554839 27901 raft_consensus.cc:2272] T d7574b0cc43f4903a703314b82b0e9da P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.557296 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.557956 27901 raft_consensus.cc:2243] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.558676 27901 raft_consensus.cc:2272] T 8f1d609ecf684c38b48d776083ce2417 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.558712  3323 consensus_peers.cc:597] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.560781 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.561379 27901 raft_consensus.cc:2243] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.561812 27901 raft_consensus.cc:2272] T 5d04edf84bba4d759de41bab8b424463 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.563853 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.564515 27901 raft_consensus.cc:2243] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.565656 27901 raft_consensus.cc:2272] T a1128833b31748d0b098825fd4212f82 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.568372 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.569195 27901 raft_consensus.cc:2243] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.570218 27901 raft_consensus.cc:2272] T 22507d010a9a43f6834e5a2dfd53c148 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.572721 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.573367 27901 raft_consensus.cc:2243] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.573915 27901 raft_consensus.cc:2272] T 5557da3aee9b41f793515bea5f1f42cc P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.576753 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.577406 27901 raft_consensus.cc:2243] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.577863 27901 raft_consensus.cc:2272] T 018917aa98ef46b4990e809f93148c34 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.580235 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.580843 27901 raft_consensus.cc:2243] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.581382 27901 raft_consensus.cc:2272] T ecac352d2640492ba17f310cbb649d7c P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.584254 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.584836 27901 raft_consensus.cc:2243] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.585184 27901 raft_consensus.cc:2272] T 98065e64beff449b8caef22a3f2a49f5 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.587073 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.587642 27901 raft_consensus.cc:2243] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.588135 27901 raft_consensus.cc:2272] T 7c16129126724cb3a6e2ecfa3547b4e6 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.590306 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.590842 27901 raft_consensus.cc:2243] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.591292 27901 raft_consensus.cc:2272] T e514dae1f1814a2b80b772b07399cc58 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.593883 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.594684 27901 raft_consensus.cc:2243] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.595836 27901 raft_consensus.cc:2272] T 78ac2072e92b45fd89cb3b4057fb9e7b P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.597750  3397 consensus_peers.cc:597] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.598747 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.599315 27901 raft_consensus.cc:2243] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.599746 27901 raft_consensus.cc:2272] T a1537c0aaa1d470e930915f178ec10d2 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.601748 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.602316 27901 raft_consensus.cc:2243] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.602662 27901 raft_consensus.cc:2272] T d732d672e75142f79a290bd7f5ce0840 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.604624 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.605266 27901 raft_consensus.cc:2243] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.605712 27901 raft_consensus.cc:2272] T 27a5322a6cb9422f85206f14c490fb6a P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.608388 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.609269 27901 raft_consensus.cc:2243] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.609772 27901 raft_consensus.cc:2272] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.610766  3471 consensus_peers.cc:597] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.611995 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.612720 27901 raft_consensus.cc:2243] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.613821 27901 raft_consensus.cc:2272] T b88e30e8276a41a98a3e4f6cf043375a P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.616669 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.617195 27901 raft_consensus.cc:2243] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.617583 27901 raft_consensus.cc:2272] T 304d003e5a044246b2968b09bafc2146 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.620069 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.620816 27901 raft_consensus.cc:2243] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.621261 27901 raft_consensus.cc:2272] T f05be967181241fdb7f127f3505bbebc P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.623410 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.624202 27901 raft_consensus.cc:2243] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.625102 27901 raft_consensus.cc:2272] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.627354 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.627916 27901 raft_consensus.cc:2243] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.628679 27901 raft_consensus.cc:2272] T 8ae6143fac5245deabc5979617a4f018 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.630749 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.631434 27901 raft_consensus.cc:2243] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.632346 27901 raft_consensus.cc:2272] T f7e3b18f25574f46ac56a124bb52c1c3 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.634261 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.634769 27901 raft_consensus.cc:2243] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.635190 27901 raft_consensus.cc:2272] T 40971e1ab1f04b21a166d7eac0ee7d95 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.637048 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.637537 27901 raft_consensus.cc:2243] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.638348 27901 raft_consensus.cc:2272] T c4e38e5f24604ea48097687a200d7e48 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.641147 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.641830 27901 raft_consensus.cc:2243] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.642319 27901 raft_consensus.cc:2272] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.644593 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.645131 27901 raft_consensus.cc:2243] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.645525 27901 raft_consensus.cc:2272] T 94c6fcbbfd7244c0a66649594e57fb03 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.648140 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.648789 27901 raft_consensus.cc:2243] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.649367 27901 raft_consensus.cc:2272] T f9ccb9fa24984b6c9a11b8429cd45338 P 972a4469a4654416ab0a153d8c740282 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.650419  3471 consensus_peers.cc:597] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.651517 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.652259 27901 raft_consensus.cc:2243] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.652623 27901 raft_consensus.cc:2272] T a468aba2a1664f7b8a5e1755775a2e72 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.654660 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.655130 27901 raft_consensus.cc:2243] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.655560 27901 raft_consensus.cc:2272] T e3a384f005d84258a11b79619f7d7ef4 P 972a4469a4654416ab0a153d8c740282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.657706 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:25.658233 27901 raft_consensus.cc:2243] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.658751 27901 raft_consensus.cc:2272] T f058e8e6fdfa480db59f5004728b8f80 P 972a4469a4654416ab0a153d8c740282 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.667965  3397 consensus_peers.cc:597] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.669308  3323 consensus_peers.cc:597] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.695104  3471 consensus_peers.cc:597] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.728264  3471 consensus_peers.cc:597] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.729315 27901 tablet_server.cc:196] TabletServer@127.27.63.65:34091 shutdown complete.
W20260709 13:14:25.731086  3397 consensus_peers.cc:597] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.755836  3323 consensus_peers.cc:597] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.760830  3397 consensus_peers.cc:597] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.765332  3397 consensus_peers.cc:597] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.796240  3471 consensus_peers.cc:597] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.796350  3323 consensus_peers.cc:597] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.801319  3471 consensus_peers.cc:597] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.819428 27901 tablet_server.cc:179] TabletServer@127.27.63.66:0 shutting down...
W20260709 13:14:25.827189  3323 consensus_peers.cc:597] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.841209  3471 consensus_peers.cc:597] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.849098  3471 consensus_peers.cc:597] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.858510  3470 consensus_peers.cc:597] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.862820  3471 consensus_peers.cc:597] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.870217  3397 consensus_peers.cc:597] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.871695  3323 consensus_peers.cc:597] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:25.871866  3396 consensus_peers.cc:597] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.876441 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:25.877257 27901 tablet_replica.cc:333] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.878019 27901 raft_consensus.cc:2243] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.878613 27901 raft_consensus.cc:2272] T 03a915f11e804f04adb461457918ee75 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.881299 27901 tablet_replica.cc:333] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.882042 27901 raft_consensus.cc:2243] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.882930 27901 raft_consensus.cc:2272] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.883028  3397 consensus_peers.cc:597] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.885119 27901 tablet_replica.cc:333] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.885737 27901 raft_consensus.cc:2243] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.886164 27901 raft_consensus.cc:2272] T 63dbef5a847a4451b6e4e46d15341334 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.888898 27901 tablet_replica.cc:333] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.889750 27901 raft_consensus.cc:2243] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.890295 27901 raft_consensus.cc:2272] T 85719491f2464bbf9c92b2f19238beeb P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.893364 27901 tablet_replica.cc:333] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.894217 27901 raft_consensus.cc:2243] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.895103 27901 raft_consensus.cc:2272] T ad53e24ea0f24ba6977b81fbb4608a83 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.897663 27901 tablet_replica.cc:333] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.898253 27901 raft_consensus.cc:2243] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.898658 27901 raft_consensus.cc:2272] T e1bb6275bf5a44debe9b7cdd0d757a3e P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.900794 27901 tablet_replica.cc:333] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.901472 27901 raft_consensus.cc:2243] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.901921 27901 raft_consensus.cc:2272] T 7c16129126724cb3a6e2ecfa3547b4e6 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.903755  3470 consensus_peers.cc:597] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.904418 27901 tablet_replica.cc:333] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.904994 27901 raft_consensus.cc:2243] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.905426 27901 raft_consensus.cc:2272] T 8ae6143fac5245deabc5979617a4f018 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.907944 27901 tablet_replica.cc:333] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.908692 27901 raft_consensus.cc:2243] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.909547 27901 raft_consensus.cc:2272] T 596e0222214f40b5861691b7bcaeea24 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.911563 27901 tablet_replica.cc:333] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
W20260709 13:14:25.911825  3323 consensus_peers.cc:597] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.912264 27901 raft_consensus.cc:2243] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:25.912329  3323 consensus_peers.cc:597] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.912902 27901 raft_consensus.cc:2272] T b61ff345dd8d4467829eded70f16f6a0 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.915182 27901 tablet_replica.cc:333] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.915743 27901 raft_consensus.cc:2243] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.916224 27901 raft_consensus.cc:2272] T f7e3b18f25574f46ac56a124bb52c1c3 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.917912 27901 tablet_replica.cc:333] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.918452 27901 raft_consensus.cc:2243] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.918867 27901 raft_consensus.cc:2272] T e514dae1f1814a2b80b772b07399cc58 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.920876 27901 tablet_replica.cc:333] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.921413 27901 raft_consensus.cc:2243] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.921813 27901 raft_consensus.cc:2272] T 8051a5cc00534c788d643e0b21d3d9c2 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.923992 27901 tablet_replica.cc:333] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.924858 27901 raft_consensus.cc:2243] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:14:25.925318  3323 consensus_peers.cc:597] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.925902 27901 raft_consensus.cc:2272] T 9f6db7a8007940d6b8bc088af66589ae P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.927961  3323 consensus_peers.cc:597] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.928663 27901 tablet_replica.cc:333] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.929327 27901 raft_consensus.cc:2243] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.929755 27901 raft_consensus.cc:2272] T f2794d9308d0458aa11c33cb7dab1748 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.931900 27901 tablet_replica.cc:333] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.932511 27901 raft_consensus.cc:2243] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.932934 27901 raft_consensus.cc:2272] T bf2d92deaf5c4766a2cc54da4c274434 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.934912 27901 tablet_replica.cc:333] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.935467 27901 raft_consensus.cc:2243] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.936298 27901 raft_consensus.cc:2272] T 27a5322a6cb9422f85206f14c490fb6a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.938203 27901 tablet_replica.cc:333] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.938748 27901 raft_consensus.cc:2243] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.939507 27901 raft_consensus.cc:2272] T a1537c0aaa1d470e930915f178ec10d2 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.941493 27901 tablet_replica.cc:333] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.942030 27901 raft_consensus.cc:2243] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.942440 27901 raft_consensus.cc:2272] T d732d672e75142f79a290bd7f5ce0840 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.944370 27901 tablet_replica.cc:333] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.944922 27901 raft_consensus.cc:2243] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.945315 27901 raft_consensus.cc:2272] T beb78494b2e0410ba8b06a30fff7158a P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.947101 27901 tablet_replica.cc:333] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.947631 27901 raft_consensus.cc:2243] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.948467 27901 raft_consensus.cc:2272] T 94c6fcbbfd7244c0a66649594e57fb03 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.950340 27901 tablet_replica.cc:333] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.950855 27901 raft_consensus.cc:2243] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.951280 27901 raft_consensus.cc:2272] T 40971e1ab1f04b21a166d7eac0ee7d95 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.953157 27901 tablet_replica.cc:333] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.953655 27901 raft_consensus.cc:2243] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.954054 27901 raft_consensus.cc:2272] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.955811 27901 tablet_replica.cc:333] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.956370 27901 raft_consensus.cc:2243] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.956826 27901 raft_consensus.cc:2272] T 208caff3bfdf46a495a86f06a667659a P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.958529 27901 tablet_replica.cc:333] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.959022 27901 raft_consensus.cc:2243] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.959422 27901 raft_consensus.cc:2272] T 691ec311298b4d19a1e62763ea313587 P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.961755 27901 tablet_replica.cc:333] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.962276 27901 raft_consensus.cc:2243] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.963042 27901 raft_consensus.cc:2272] T ecac352d2640492ba17f310cbb649d7c P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.964838  3471 consensus_peers.cc:597] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.964869 27901 tablet_replica.cc:333] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.965910 27901 raft_consensus.cc:2243] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.966785 27901 raft_consensus.cc:2272] T 98065e64beff449b8caef22a3f2a49f5 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.969007 27901 tablet_replica.cc:333] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.969843 27901 raft_consensus.cc:2243] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.970394 27901 raft_consensus.cc:2272] T 65d7199324524f1fbb9a40b53925a3a1 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.973011 27901 tablet_replica.cc:333] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.973662 27901 raft_consensus.cc:2243] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.974694 27901 raft_consensus.cc:2272] T 018917aa98ef46b4990e809f93148c34 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.977684 27901 tablet_replica.cc:333] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.978442 27901 raft_consensus.cc:2243] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.978928 27901 raft_consensus.cc:2272] T 5d04edf84bba4d759de41bab8b424463 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:25.980232  3471 consensus_peers.cc:597] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.981690 27901 tablet_replica.cc:333] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.982770 27901 raft_consensus.cc:2243] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.983284 27901 raft_consensus.cc:2272] T f23621b920b44c8dbdd783bed09dfa50 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.985430 27901 tablet_replica.cc:333] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.986013 27901 raft_consensus.cc:2243] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.986470 27901 raft_consensus.cc:2272] T a468aba2a1664f7b8a5e1755775a2e72 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.988382 27901 tablet_replica.cc:333] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.988978 27901 raft_consensus.cc:2243] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.989385 27901 raft_consensus.cc:2272] T f058e8e6fdfa480db59f5004728b8f80 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.991479 27901 tablet_replica.cc:333] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:25.992233 27901 raft_consensus.cc:2243] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:25.993218 27901 raft_consensus.cc:2272] T c49acd4ec7dc42a8b9a1cb2835d1bede P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.995718 27901 tablet_replica.cc:333] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
W20260709 13:14:25.995764  3396 consensus_peers.cc:597] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.996531 27901 raft_consensus.cc:2243] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:25.997136 27901 raft_consensus.cc:2272] T 22507d010a9a43f6834e5a2dfd53c148 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:25.999850 27901 tablet_replica.cc:333] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.000412 27901 raft_consensus.cc:2243] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.001214 27901 raft_consensus.cc:2272] T 5557da3aee9b41f793515bea5f1f42cc P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:26.002394  3397 consensus_peers.cc:597] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea -> Peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Couldn't send request to peer 972a4469a4654416ab0a153d8c740282. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.003377 27901 tablet_replica.cc:333] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.003964 27901 raft_consensus.cc:2243] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.004458 27901 raft_consensus.cc:2272] T 304d003e5a044246b2968b09bafc2146 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.006291 27901 tablet_replica.cc:333] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.006806 27901 raft_consensus.cc:2243] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.007237 27901 raft_consensus.cc:2272] T b88e30e8276a41a98a3e4f6cf043375a P 83ef2126cf6d4f628cd668d0817871f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.009183 27901 tablet_replica.cc:333] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.009682 27901 raft_consensus.cc:2243] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.010092 27901 raft_consensus.cc:2272] T d5cbb74eb76044f2a65c7a0b52c2adde P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.012197 27901 tablet_replica.cc:333] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.012820 27901 raft_consensus.cc:2243] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.013300 27901 raft_consensus.cc:2272] T 3061a90b8dbc42488cec94032788e597 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.016075 27901 tablet_replica.cc:333] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.016775 27901 raft_consensus.cc:2243] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.017457 27901 raft_consensus.cc:2272] T 59159d79fbfc4835a64e52a2783f46f7 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.019965 27901 tablet_replica.cc:333] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.020673 27901 raft_consensus.cc:2243] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.021615 27901 raft_consensus.cc:2272] T 094d0615636b4da4a2c8ba92b4218624 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.023957 27901 tablet_replica.cc:333] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.024591 27901 raft_consensus.cc:2243] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.025311 27901 raft_consensus.cc:2272] T f4611ffabfbe4fa39f2fa67f79c3b048 P 83ef2126cf6d4f628cd668d0817871f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.027160 27901 tablet_replica.cc:333] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2: stopping tablet replica
I20260709 13:14:26.027683 27901 raft_consensus.cc:2243] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.028133 27901 raft_consensus.cc:2272] T 514a52a2733c4c2f996475e12c2dbaa8 P 83ef2126cf6d4f628cd668d0817871f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:26.041530  3396 consensus_peers.cc:597] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.067917  3396 consensus_peers.cc:597] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.073139  3396 consensus_peers.cc:597] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.099735 27901 tablet_server.cc:196] TabletServer@127.27.63.66:0 shutdown complete.
W20260709 13:14:26.122315  3396 consensus_peers.cc:597] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.122906  3396 consensus_peers.cc:597] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.146839  3470 consensus_peers.cc:597] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.160774  3396 consensus_peers.cc:597] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.178442  3470 consensus_peers.cc:597] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.179116  3470 consensus_peers.cc:597] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.188318  3470 consensus_peers.cc:597] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.193305 27901 tablet_server.cc:179] TabletServer@127.27.63.67:0 shutting down...
W20260709 13:14:26.201607  3396 consensus_peers.cc:597] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.218650  3470 consensus_peers.cc:597] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.222126  3470 consensus_peers.cc:597] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.226241  3470 consensus_peers.cc:597] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.232241  3396 consensus_peers.cc:597] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.247390  3470 consensus_peers.cc:597] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.249593  3470 consensus_peers.cc:597] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.267593 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:26.268400 27901 tablet_replica.cc:333] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.269035 27901 raft_consensus.cc:2243] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.269608 27901 raft_consensus.cc:2272] T f05be967181241fdb7f127f3505bbebc P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.271865 27901 tablet_replica.cc:333] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.272500 27901 raft_consensus.cc:2243] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.272998 27901 raft_consensus.cc:2272] T 22507d010a9a43f6834e5a2dfd53c148 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.275141 27901 tablet_replica.cc:333] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.275853 27901 raft_consensus.cc:2243] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.276820 27901 raft_consensus.cc:2272] T 03a915f11e804f04adb461457918ee75 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.279480 27901 tablet_replica.cc:333] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.280287 27901 raft_consensus.cc:2243] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.280850 27901 raft_consensus.cc:2272] T 63dbef5a847a4451b6e4e46d15341334 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.283857 27901 tablet_replica.cc:333] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.284981 27901 raft_consensus.cc:2243] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.285552 27901 raft_consensus.cc:2272] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:26.286688  3470 consensus_peers.cc:597] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.289026 27901 tablet_replica.cc:333] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
W20260709 13:14:26.283769  3396 consensus_peers.cc:597] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.289944 27901 raft_consensus.cc:2243] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.290540 27901 raft_consensus.cc:2272] T 85719491f2464bbf9c92b2f19238beeb P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.293213 27901 tablet_replica.cc:333] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.294024 27901 raft_consensus.cc:2243] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.294580 27901 raft_consensus.cc:2272] T f9ccb9fa24984b6c9a11b8429cd45338 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:26.296742  3470 consensus_peers.cc:597] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.297094 27901 tablet_replica.cc:333] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.297787 27901 raft_consensus.cc:2243] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.298236 27901 raft_consensus.cc:2272] T a9d414c916bc43f2a6579cf5fa1beaee P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.300729 27901 tablet_replica.cc:333] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.301508 27901 raft_consensus.cc:2243] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.301999 27901 raft_consensus.cc:2272] T ad53e24ea0f24ba6977b81fbb4608a83 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.304119 27901 tablet_replica.cc:333] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.304612 27901 raft_consensus.cc:2243] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.305276 27901 raft_consensus.cc:2272] T e1bb6275bf5a44debe9b7cdd0d757a3e P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.307060 27901 tablet_replica.cc:333] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.307528 27901 raft_consensus.cc:2243] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.307904 27901 raft_consensus.cc:2272] T a1128833b31748d0b098825fd4212f82 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.309703 27901 tablet_replica.cc:333] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.310168 27901 raft_consensus.cc:2243] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.310837 27901 raft_consensus.cc:2272] T bb18f85836cc456b8362bb08424827ca P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.312681 27901 tablet_replica.cc:333] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.313293 27901 raft_consensus.cc:2243] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.313776 27901 raft_consensus.cc:2272] T 8ae6143fac5245deabc5979617a4f018 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.316113 27901 tablet_replica.cc:333] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
W20260709 13:14:26.316675  3396 consensus_peers.cc:597] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.316794 27901 raft_consensus.cc:2243] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.317446 27901 raft_consensus.cc:2272] T 596e0222214f40b5861691b7bcaeea24 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.319423 27901 tablet_replica.cc:333] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.319949 27901 raft_consensus.cc:2243] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.320422 27901 raft_consensus.cc:2272] T a1537c0aaa1d470e930915f178ec10d2 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.322224 27901 tablet_replica.cc:333] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.322721 27901 raft_consensus.cc:2243] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.323419 27901 raft_consensus.cc:2272] T d732d672e75142f79a290bd7f5ce0840 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.326152 27901 tablet_replica.cc:333] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.326906 27901 raft_consensus.cc:2243] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.327909 27901 raft_consensus.cc:2272] T 65d7199324524f1fbb9a40b53925a3a1 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.330497 27901 tablet_replica.cc:333] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.331254 27901 raft_consensus.cc:2243] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.331892 27901 raft_consensus.cc:2272] T 018917aa98ef46b4990e809f93148c34 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:26.333689  3396 consensus_peers.cc:597] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea -> Peer 83ef2126cf6d4f628cd668d0817871f2 (127.27.63.66:33623): Couldn't send request to peer 83ef2126cf6d4f628cd668d0817871f2. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:33623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.334658 27901 tablet_replica.cc:333] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.335340 27901 raft_consensus.cc:2243] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.336282 27901 raft_consensus.cc:2272] T f2794d9308d0458aa11c33cb7dab1748 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:26.336643  3470 consensus_peers.cc:597] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.338406 27901 tablet_replica.cc:333] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.338960 27901 raft_consensus.cc:2243] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.339341 27901 raft_consensus.cc:2272] T bf2d92deaf5c4766a2cc54da4c274434 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.341207 27901 tablet_replica.cc:333] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.341686 27901 raft_consensus.cc:2243] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.342361 27901 raft_consensus.cc:2272] T 514a52a2733c4c2f996475e12c2dbaa8 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.344218 27901 tablet_replica.cc:333] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.344774 27901 raft_consensus.cc:2243] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.345455 27901 raft_consensus.cc:2272] T e514dae1f1814a2b80b772b07399cc58 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.347182 27901 tablet_replica.cc:333] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.347682 27901 raft_consensus.cc:2243] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.348407 27901 raft_consensus.cc:2272] T 8051a5cc00534c788d643e0b21d3d9c2 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.350168 27901 tablet_replica.cc:333] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.350828 27901 raft_consensus.cc:2243] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.351367 27901 raft_consensus.cc:2272] T 78ac2072e92b45fd89cb3b4057fb9e7b P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.354117 27901 tablet_replica.cc:333] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.354591 27901 raft_consensus.cc:2243] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.354939 27901 raft_consensus.cc:2272] T 208caff3bfdf46a495a86f06a667659a P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.356779 27901 tablet_replica.cc:333] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.357252 27901 raft_consensus.cc:2243] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.357935 27901 raft_consensus.cc:2272] T 691ec311298b4d19a1e62763ea313587 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.359751 27901 tablet_replica.cc:333] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.360427 27901 raft_consensus.cc:2243] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.360929 27901 raft_consensus.cc:2272] T 9042f5a22d47425eac26b6f03be99d9d P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.363456 27901 tablet_replica.cc:333] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.364205 27901 raft_consensus.cc:2243] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.364727 27901 raft_consensus.cc:2272] T beb78494b2e0410ba8b06a30fff7158a P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.367139 27901 tablet_replica.cc:333] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.367969 27901 raft_consensus.cc:2243] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.368568 27901 raft_consensus.cc:2272] T b61ff345dd8d4467829eded70f16f6a0 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.371356 27901 tablet_replica.cc:333] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.372105 27901 raft_consensus.cc:2243] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.372658 27901 raft_consensus.cc:2272] T 9f6db7a8007940d6b8bc088af66589ae P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.375377 27901 tablet_replica.cc:333] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.376209 27901 raft_consensus.cc:2243] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.376781 27901 raft_consensus.cc:2272] T 87e493d066c44520bd444d8afc63ec91 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.378940 27901 tablet_replica.cc:333] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.379455 27901 raft_consensus.cc:2243] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.380300 27901 raft_consensus.cc:2272] T 3061a90b8dbc42488cec94032788e597 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.382246 27901 tablet_replica.cc:333] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.382750 27901 raft_consensus.cc:2243] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.383468 27901 raft_consensus.cc:2272] T 361bb4036c344c3cbf83650eeaf091aa P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.385352 27901 tablet_replica.cc:333] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.385833 27901 raft_consensus.cc:2243] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.386245 27901 raft_consensus.cc:2272] T c49acd4ec7dc42a8b9a1cb2835d1bede P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.388139 27901 tablet_replica.cc:333] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.388630 27901 raft_consensus.cc:2243] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.389366 27901 raft_consensus.cc:2272] T 304d003e5a044246b2968b09bafc2146 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.391243 27901 tablet_replica.cc:333] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.391739 27901 raft_consensus.cc:2243] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.392195 27901 raft_consensus.cc:2272] T d5cbb74eb76044f2a65c7a0b52c2adde P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.393913 27901 tablet_replica.cc:333] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.394399 27901 raft_consensus.cc:2243] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.395118 27901 raft_consensus.cc:2272] T f23621b920b44c8dbdd783bed09dfa50 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.396967 27901 tablet_replica.cc:333] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.397435 27901 raft_consensus.cc:2243] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.397819 27901 raft_consensus.cc:2272] T e3a384f005d84258a11b79619f7d7ef4 P 06964934536148c2aa77a2f1c6382fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.399593 27901 tablet_replica.cc:333] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.400118 27901 raft_consensus.cc:2243] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.400876 27901 raft_consensus.cc:2272] T f058e8e6fdfa480db59f5004728b8f80 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.402652 27901 tablet_replica.cc:333] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.403124 27901 raft_consensus.cc:2243] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.403529 27901 raft_consensus.cc:2272] T 8f1d609ecf684c38b48d776083ce2417 P 06964934536148c2aa77a2f1c6382fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.405328 27901 tablet_replica.cc:333] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.405869 27901 raft_consensus.cc:2243] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.406327 27901 raft_consensus.cc:2272] T d7574b0cc43f4903a703314b82b0e9da P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.408617 27901 tablet_replica.cc:333] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.409169 27901 raft_consensus.cc:2243] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.409667 27901 raft_consensus.cc:2272] T d1ff273fcfdc45a386458728eb7254f3 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.412034 27901 tablet_replica.cc:333] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.412637 27901 raft_consensus.cc:2243] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.413117 27901 raft_consensus.cc:2272] T c4e38e5f24604ea48097687a200d7e48 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.415504 27901 tablet_replica.cc:333] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea: stopping tablet replica
I20260709 13:14:26.415972 27901 raft_consensus.cc:2243] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.416446 27901 raft_consensus.cc:2272] T f4611ffabfbe4fa39f2fa67f79c3b048 P 06964934536148c2aa77a2f1c6382fea [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:26.432826  3470 consensus_peers.cc:597] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.441073  3470 consensus_peers.cc:597] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:26.447996  3470 consensus_peers.cc:597] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.476337 27901 tablet_server.cc:196] TabletServer@127.27.63.67:0 shutdown complete.
I20260709 13:14:26.541447 27901 tablet_server.cc:179] TabletServer@127.27.63.68:0 shutting down...
I20260709 13:14:26.574618  4494 raft_consensus.cc:493] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 972a4469a4654416ab0a153d8c740282)
I20260709 13:14:26.575212  4494 raft_consensus.cc:515] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06964934536148c2aa77a2f1c6382fea" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 32907 } } peers { permanent_uuid: "73143d6f16c94a3aaef99f8e07588c1f" member_type: VOTER last_known_addr { host: "127.27.63.68" port: 33961 } } peers { permanent_uuid: "972a4469a4654416ab0a153d8c740282" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 34091 } }
I20260709 13:14:26.577036  4494 leader_election.cc:290] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907), 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091)
W20260709 13:14:26.581429  3470 leader_election.cc:336] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Network error: Client connection negotiation failed: client connection to 127.27.63.67:32907: connect: Connection refused (error 111)
W20260709 13:14:26.581943  3471 leader_election.cc:336] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 972a4469a4654416ab0a153d8c740282 (127.27.63.65:34091): Network error: Client connection negotiation failed: client connection to 127.27.63.65:34091: connect: Connection refused (error 111)
I20260709 13:14:26.582397  3471 leader_election.cc:304] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73143d6f16c94a3aaef99f8e07588c1f; no voters: 06964934536148c2aa77a2f1c6382fea, 972a4469a4654416ab0a153d8c740282
I20260709 13:14:26.583217  4494 raft_consensus.cc:2749] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:26.586864 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:26.588338 27901 tablet_replica.cc:333] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.589043 27901 raft_consensus.cc:2243] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.589553 27901 raft_consensus.cc:2272] T 03a915f11e804f04adb461457918ee75 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.592350 27901 tablet_replica.cc:333] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.593096 27901 raft_consensus.cc:2243] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.594609 27901 raft_consensus.cc:2272] T 6d122fabfab34d9ea6e8bf2100aaf1e3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.597335 27901 tablet_replica.cc:333] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.597985 27901 raft_consensus.cc:2243] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:14:26.598222  3470 consensus_peers.cc:597] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.67:32907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.598915 27901 raft_consensus.cc:2272] T f05be967181241fdb7f127f3505bbebc P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.600991 27901 tablet_replica.cc:333] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.601550 27901 raft_consensus.cc:2243] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.602252 27901 raft_consensus.cc:2272] T 63dbef5a847a4451b6e4e46d15341334 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.604313 27901 tablet_replica.cc:333] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.604821 27901 raft_consensus.cc:2243] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.605172 27901 raft_consensus.cc:2272] T 27a5322a6cb9422f85206f14c490fb6a P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.607015 27901 tablet_replica.cc:333] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.607574 27901 raft_consensus.cc:2243] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.608337 27901 raft_consensus.cc:2272] T 85719491f2464bbf9c92b2f19238beeb P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.610190 27901 tablet_replica.cc:333] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.610708 27901 raft_consensus.cc:2243] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.611078 27901 raft_consensus.cc:2272] T 5dd7e8a9e3e248c4ba469d477a27f8cb P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.613333 27901 tablet_replica.cc:333] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.614032 27901 raft_consensus.cc:2243] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.614959 27901 raft_consensus.cc:2272] T f9ccb9fa24984b6c9a11b8429cd45338 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.617724 27901 tablet_replica.cc:333] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.618558 27901 raft_consensus.cc:2243] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.619115 27901 raft_consensus.cc:2272] T a9d414c916bc43f2a6579cf5fa1beaee P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:26.621522  3470 consensus_peers.cc:597] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f -> Peer 06964934536148c2aa77a2f1c6382fea (127.27.63.67:32907): Couldn't send request to peer 06964934536148c2aa77a2f1c6382fea. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.67:32907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.621891 27901 tablet_replica.cc:333] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.622503 27901 raft_consensus.cc:2243] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.622879 27901 raft_consensus.cc:2272] T ad53e24ea0f24ba6977b81fbb4608a83 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.624883 27901 tablet_replica.cc:333] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.625384 27901 raft_consensus.cc:2243] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.626060 27901 raft_consensus.cc:2272] T 7c16129126724cb3a6e2ecfa3547b4e6 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.627885 27901 tablet_replica.cc:333] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.628463 27901 raft_consensus.cc:2243] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.628846 27901 raft_consensus.cc:2272] T 094d0615636b4da4a2c8ba92b4218624 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.630498 27901 tablet_replica.cc:333] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.630959 27901 raft_consensus.cc:2243] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.631301 27901 raft_consensus.cc:2272] T 596e0222214f40b5861691b7bcaeea24 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.633033 27901 tablet_replica.cc:333] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.633507 27901 raft_consensus.cc:2243] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.633854 27901 raft_consensus.cc:2272] T e1bb6275bf5a44debe9b7cdd0d757a3e P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.635680 27901 tablet_replica.cc:333] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.636214 27901 raft_consensus.cc:2243] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.636631 27901 raft_consensus.cc:2272] T f7e3b18f25574f46ac56a124bb52c1c3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.638501 27901 tablet_replica.cc:333] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.639021 27901 raft_consensus.cc:2243] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.639418 27901 raft_consensus.cc:2272] T ecac352d2640492ba17f310cbb649d7c P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.641184 27901 tablet_replica.cc:333] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.641734 27901 raft_consensus.cc:2243] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.642153 27901 raft_consensus.cc:2272] T 98065e64beff449b8caef22a3f2a49f5 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.643989 27901 tablet_replica.cc:333] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.644574 27901 raft_consensus.cc:2243] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.644975 27901 raft_consensus.cc:2272] T 65d7199324524f1fbb9a40b53925a3a1 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.646723 27901 tablet_replica.cc:333] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.647251 27901 raft_consensus.cc:2243] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.648023 27901 raft_consensus.cc:2272] T 87e493d066c44520bd444d8afc63ec91 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.649866 27901 tablet_replica.cc:333] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.650388 27901 raft_consensus.cc:2243] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.650821 27901 raft_consensus.cc:2272] T 8f1d609ecf684c38b48d776083ce2417 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.653053 27901 tablet_replica.cc:333] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.653647 27901 raft_consensus.cc:2243] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.654523 27901 raft_consensus.cc:2272] T d7574b0cc43f4903a703314b82b0e9da P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.656790 27901 tablet_replica.cc:333] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.657262 27901 raft_consensus.cc:2243] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.657648 27901 raft_consensus.cc:2272] T 9042f5a22d47425eac26b6f03be99d9d P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.659363 27901 tablet_replica.cc:333] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.659840 27901 raft_consensus.cc:2243] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.660619 27901 raft_consensus.cc:2272] T beb78494b2e0410ba8b06a30fff7158a P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.662402 27901 tablet_replica.cc:333] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.662911 27901 raft_consensus.cc:2243] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.663249 27901 raft_consensus.cc:2272] T bb18f85836cc456b8362bb08424827ca P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.665035 27901 tablet_replica.cc:333] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.665594 27901 raft_consensus.cc:2243] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.665954 27901 raft_consensus.cc:2272] T 59159d79fbfc4835a64e52a2783f46f7 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.667870 27901 tablet_replica.cc:333] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.668486 27901 raft_consensus.cc:2243] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.669195 27901 raft_consensus.cc:2272] T d5cbb74eb76044f2a65c7a0b52c2adde P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.671066 27901 tablet_replica.cc:333] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.671554 27901 raft_consensus.cc:2243] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.671921 27901 raft_consensus.cc:2272] T b88e30e8276a41a98a3e4f6cf043375a P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.673894 27901 tablet_replica.cc:333] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.674368 27901 raft_consensus.cc:2243] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.674737 27901 raft_consensus.cc:2272] T 514a52a2733c4c2f996475e12c2dbaa8 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.676656 27901 tablet_replica.cc:333] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.677165 27901 raft_consensus.cc:2243] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.677505 27901 raft_consensus.cc:2272] T 8051a5cc00534c788d643e0b21d3d9c2 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.679140 27901 tablet_replica.cc:333] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.679616 27901 raft_consensus.cc:2243] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.679952 27901 raft_consensus.cc:2272] T 78ac2072e92b45fd89cb3b4057fb9e7b P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.681793 27901 tablet_replica.cc:333] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.682256 27901 raft_consensus.cc:2243] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.682610 27901 raft_consensus.cc:2272] T f23621b920b44c8dbdd783bed09dfa50 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.684374 27901 tablet_replica.cc:333] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.684870 27901 raft_consensus.cc:2243] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.685554 27901 raft_consensus.cc:2272] T e3a384f005d84258a11b79619f7d7ef4 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.687312 27901 tablet_replica.cc:333] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.687814 27901 raft_consensus.cc:2243] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.688568 27901 raft_consensus.cc:2272] T a468aba2a1664f7b8a5e1755775a2e72 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.690315 27901 tablet_replica.cc:333] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.690785 27901 raft_consensus.cc:2243] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.691556 27901 raft_consensus.cc:2272] T 40971e1ab1f04b21a166d7eac0ee7d95 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.693274 27901 tablet_replica.cc:333] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.693735 27901 raft_consensus.cc:2243] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.694075 27901 raft_consensus.cc:2272] T c4e38e5f24604ea48097687a200d7e48 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.695930 27901 tablet_replica.cc:333] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.696461 27901 raft_consensus.cc:2243] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.697134 27901 raft_consensus.cc:2272] T 46e6844fd8644c6498d5b9ce0e7cbd6b P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.698899 27901 tablet_replica.cc:333] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.699397 27901 raft_consensus.cc:2243] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.699753 27901 raft_consensus.cc:2272] T 94c6fcbbfd7244c0a66649594e57fb03 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.701519 27901 tablet_replica.cc:333] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.701936 27901 raft_consensus.cc:2243] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.702373 27901 raft_consensus.cc:2272] T 691ec311298b4d19a1e62763ea313587 P 73143d6f16c94a3aaef99f8e07588c1f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.704118 27901 tablet_replica.cc:333] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.704605 27901 raft_consensus.cc:2243] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.704954 27901 raft_consensus.cc:2272] T a1128833b31748d0b098825fd4212f82 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.706601 27901 tablet_replica.cc:333] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.707031 27901 raft_consensus.cc:2243] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.707716 27901 raft_consensus.cc:2272] T 5d04edf84bba4d759de41bab8b424463 P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.709517 27901 tablet_replica.cc:333] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.709970 27901 raft_consensus.cc:2243] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.710448 27901 raft_consensus.cc:2272] T c49acd4ec7dc42a8b9a1cb2835d1bede P 73143d6f16c94a3aaef99f8e07588c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.712244 27901 tablet_replica.cc:333] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.712641 27901 raft_consensus.cc:2243] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.712971 27901 raft_consensus.cc:2272] T 361bb4036c344c3cbf83650eeaf091aa P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.714587 27901 tablet_replica.cc:333] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.714975 27901 raft_consensus.cc:2243] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.715737 27901 raft_consensus.cc:2272] T d1ff273fcfdc45a386458728eb7254f3 P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.717555 27901 tablet_replica.cc:333] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f: stopping tablet replica
I20260709 13:14:26.717937 27901 raft_consensus.cc:2243] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:26.718294 27901 raft_consensus.cc:2272] T 5557da3aee9b41f793515bea5f1f42cc P 73143d6f16c94a3aaef99f8e07588c1f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.782902 27901 tablet_server.cc:196] TabletServer@127.27.63.68:0 shutdown complete.
I20260709 13:14:26.844486 27901 master.cc:562] Master@127.27.63.126:36891 shutting down...
I20260709 13:14:26.870826 27901 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:26.871332 27901 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:26.871608 27901 tablet_replica.cc:333] T 00000000000000000000000000000000 P 48e1bd57ba224cbeb8fe214f140e07da: stopping tablet replica
I20260709 13:14:26.892989 27901 master.cc:584] Master@127.27.63.126:36891 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (32786 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:14:26.979945 27901 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.63.126:39157
I20260709 13:14:26.981177 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:26.986651  4497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:26.986707  4498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:26.988106  4500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:26.989375 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:26.990504 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:26.990870 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:26.991017 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602866990999 us; error 0 us; skew 500 ppm
I20260709 13:14:26.991510 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:26.994087 27901 webserver.cc:533] Webserver started at http://127.27.63.126:38623/ using document root <none> and password file <none>
I20260709 13:14:26.994524 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:26.994688 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:26.994900 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:26.995962 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/master-0-root/instance:
uuid: "e18c1581fb074325a75c30549205fd8e"
format_stamp: "Formatted at 2026-07-09 13:14:26 on dist-test-slave-sjgx"
I20260709 13:14:27.000337 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:14:27.003580  4505 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.004361 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:27.004668 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/master-0-root
uuid: "e18c1581fb074325a75c30549205fd8e"
format_stamp: "Formatted at 2026-07-09 13:14:26 on dist-test-slave-sjgx"
I20260709 13:14:27.004966 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:27.030522 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:27.031929 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:27.071496 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.126:39157
I20260709 13:14:27.071614  4556 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.126:39157 every 8 connection(s)
I20260709 13:14:27.075304  4557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.085768  4557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e: Bootstrap starting.
I20260709 13:14:27.090584  4557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.094899  4557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e: No bootstrap required, opened a new log
I20260709 13:14:27.097098  4557 raft_consensus.cc:359] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18c1581fb074325a75c30549205fd8e" member_type: VOTER }
I20260709 13:14:27.097499  4557 raft_consensus.cc:385] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.097751  4557 raft_consensus.cc:740] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18c1581fb074325a75c30549205fd8e, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.098383  4557 consensus_queue.cc:260] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18c1581fb074325a75c30549205fd8e" member_type: VOTER }
I20260709 13:14:27.098840  4557 raft_consensus.cc:399] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:27.099107  4557 raft_consensus.cc:493] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:27.099372  4557 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.103826  4557 raft_consensus.cc:515] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18c1581fb074325a75c30549205fd8e" member_type: VOTER }
I20260709 13:14:27.104354  4557 leader_election.cc:304] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e18c1581fb074325a75c30549205fd8e; no voters: 
I20260709 13:14:27.105502  4557 leader_election.cc:290] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:27.105897  4560 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:27.107297  4560 raft_consensus.cc:697] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 1 LEADER]: Becoming Leader. State: Replica: e18c1581fb074325a75c30549205fd8e, State: Running, Role: LEADER
I20260709 13:14:27.107981  4560 consensus_queue.cc:237] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18c1581fb074325a75c30549205fd8e" member_type: VOTER }
I20260709 13:14:27.108629  4557 sys_catalog.cc:565] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:27.112160  4561 sys_catalog.cc:455] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e18c1581fb074325a75c30549205fd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18c1581fb074325a75c30549205fd8e" member_type: VOTER } }
I20260709 13:14:27.112251  4562 sys_catalog.cc:455] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e18c1581fb074325a75c30549205fd8e. Latest consensus state: current_term: 1 leader_uuid: "e18c1581fb074325a75c30549205fd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18c1581fb074325a75c30549205fd8e" member_type: VOTER } }
I20260709 13:14:27.112954  4561 sys_catalog.cc:458] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:27.113231  4562 sys_catalog.cc:458] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:27.116473  4566 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:27.124565  4566 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:27.125589 27901 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:27.133009  4566 catalog_manager.cc:1383] Generated new cluster ID: 9309468bf1594d6e905548a8dec5d18f
I20260709 13:14:27.133293  4566 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:27.148538  4566 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:27.149847  4566 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:27.163169  4566 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e: Generated new TSK 0
I20260709 13:14:27.163774  4566 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:27.192564 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:27.198944  4578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:27.199894  4579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:27.203394  4581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:27.203516 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:27.204715 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:27.204891 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:27.205013 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602867205002 us; error 0 us; skew 500 ppm
I20260709 13:14:27.205474 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:27.207710 27901 webserver.cc:533] Webserver started at http://127.27.63.65:33975/ using document root <none> and password file <none>
I20260709 13:14:27.208240 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:27.208400 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:27.208607 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:27.209578 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root/instance:
uuid: "c87819469991457784c7cf663512487e"
format_stamp: "Formatted at 2026-07-09 13:14:27 on dist-test-slave-sjgx"
I20260709 13:14:27.213950 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:14:27.217157  4586 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.217912 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:27.218209 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "c87819469991457784c7cf663512487e"
format_stamp: "Formatted at 2026-07-09 13:14:27 on dist-test-slave-sjgx"
I20260709 13:14:27.218479 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:27.237701 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:27.238943 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:27.240672 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:27.242869 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:27.243036 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.243314 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:27.243467 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.282429 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:35357
I20260709 13:14:27.282523  4648 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:35357 every 8 connection(s)
I20260709 13:14:27.288019 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:27.295702  4653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:27.297403  4654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:27.299300 27901 server_base.cc:1061] running on GCE node
W20260709 13:14:27.299614  4656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:27.300688 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:27.300912 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:27.301093 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602867301077 us; error 0 us; skew 500 ppm
I20260709 13:14:27.301811 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:27.304646 27901 webserver.cc:533] Webserver started at http://127.27.63.66:35759/ using document root <none> and password file <none>
I20260709 13:14:27.305210 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:27.305169  4649 heartbeater.cc:344] Connected to a master server at 127.27.63.126:39157
I20260709 13:14:27.305641 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:27.305787  4649 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:27.305985 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:27.306730  4649 heartbeater.cc:507] Master 127.27.63.126:39157 requested a full tablet report, sending...
I20260709 13:14:27.307595 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-1-root/instance:
uuid: "86c683428f324c35a94717ce212f7276"
format_stamp: "Formatted at 2026-07-09 13:14:27 on dist-test-slave-sjgx"
I20260709 13:14:27.309051  4522 ts_manager.cc:194] Registered new tserver with Master: c87819469991457784c7cf663512487e (127.27.63.65:35357)
I20260709 13:14:27.310989  4522 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37466
I20260709 13:14:27.312676 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:14:27.316371  4661 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.317157 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:14:27.317444 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-1-root
uuid: "86c683428f324c35a94717ce212f7276"
format_stamp: "Formatted at 2026-07-09 13:14:27 on dist-test-slave-sjgx"
I20260709 13:14:27.317713 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:27.338474 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:27.339695 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:27.341145 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:27.343326 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:27.343550 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.343787 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:27.343952 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.384711 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.66:38855
I20260709 13:14:27.384819  4723 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.66:38855 every 8 connection(s)
I20260709 13:14:27.389406 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:27.396315  4727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:27.398901  4728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:27.400385  4730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:27.401028 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:27.401907 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:27.402141 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:27.402207  4724 heartbeater.cc:344] Connected to a master server at 127.27.63.126:39157
I20260709 13:14:27.402367 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602867402347 us; error 0 us; skew 500 ppm
I20260709 13:14:27.402630  4724 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:27.403138 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:27.403527  4724 heartbeater.cc:507] Master 127.27.63.126:39157 requested a full tablet report, sending...
I20260709 13:14:27.405731 27901 webserver.cc:533] Webserver started at http://127.27.63.67:34401/ using document root <none> and password file <none>
I20260709 13:14:27.405898  4522 ts_manager.cc:194] Registered new tserver with Master: 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:27.406332 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:27.406629 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:27.406909 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:27.407611  4522 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37478
I20260709 13:14:27.408179 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-2-root/instance:
uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
format_stamp: "Formatted at 2026-07-09 13:14:27 on dist-test-slave-sjgx"
I20260709 13:14:27.413122 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:14:27.416299  4735 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.416988 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:14:27.417279 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-2-root
uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
format_stamp: "Formatted at 2026-07-09 13:14:27 on dist-test-slave-sjgx"
I20260709 13:14:27.417564 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:27.436324 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:27.437554 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:27.439055 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:27.441586 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:27.441812 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.442049 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:27.442221 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:27.481132 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.67:46565
I20260709 13:14:27.481257  4797 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.67:46565 every 8 connection(s)
I20260709 13:14:27.495260  4798 heartbeater.cc:344] Connected to a master server at 127.27.63.126:39157
I20260709 13:14:27.495720  4798 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:27.496486  4798 heartbeater.cc:507] Master 127.27.63.126:39157 requested a full tablet report, sending...
I20260709 13:14:27.498366  4522 ts_manager.cc:194] Registered new tserver with Master: fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:27.498883 27901 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014553376s
I20260709 13:14:27.499776  4522 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37480
I20260709 13:14:27.524003  4522 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37492:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:27.526998  4522 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:27.610707  4613 tablet_service.cc:1511] Processing CreateTablet for tablet 82c2e3df5f3c45bf886fff877b2c8ca4 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:27.610661  4614 tablet_service.cc:1511] Processing CreateTablet for tablet f228c374a7634467987eaac91c718aac (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:27.613682  4614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f228c374a7634467987eaac91c718aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.616852  4613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c2e3df5f3c45bf886fff877b2c8ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.616868  4612 tablet_service.cc:1511] Processing CreateTablet for tablet 2529f93211584032b0ca59e2beddd84a (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:27.618495  4612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2529f93211584032b0ca59e2beddd84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.618880  4608 tablet_service.cc:1511] Processing CreateTablet for tablet 11e3af0616a047368a0ca6f6d13a72e9 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:27.621718  4610 tablet_service.cc:1511] Processing CreateTablet for tablet 5d09b12ff1334907b9032e5b613a642e (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:27.625182  4611 tablet_service.cc:1511] Processing CreateTablet for tablet dd465e5c5472416a917ba1a4bc550a44 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:27.628225  4606 tablet_service.cc:1511] Processing CreateTablet for tablet 20fcae796dda4168807069fba8c1974a (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:27.629258  4607 tablet_service.cc:1511] Processing CreateTablet for tablet e59d5484fea940eab3ee94cc6dadfea5 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:27.631515  4605 tablet_service.cc:1511] Processing CreateTablet for tablet b07adfc14da14b3faced311b3a96e2a4 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:27.617141  4609 tablet_service.cc:1511] Processing CreateTablet for tablet a813d5c1b5e5457dbf4abaa598e36f37 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:27.634392  4608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11e3af0616a047368a0ca6f6d13a72e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.637005  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd465e5c5472416a917ba1a4bc550a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.643033  4610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d09b12ff1334907b9032e5b613a642e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.648415  4606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fcae796dda4168807069fba8c1974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.649282  4813 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:27.655755  4813 tablet_bootstrap.cc:654] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.662626  4763 tablet_service.cc:1511] Processing CreateTablet for tablet f228c374a7634467987eaac91c718aac (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:27.664783  4760 tablet_service.cc:1511] Processing CreateTablet for tablet dd465e5c5472416a917ba1a4bc550a44 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:27.665479  4762 tablet_service.cc:1511] Processing CreateTablet for tablet 82c2e3df5f3c45bf886fff877b2c8ca4 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:27.668182  4761 tablet_service.cc:1511] Processing CreateTablet for tablet 2529f93211584032b0ca59e2beddd84a (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:27.670713  4759 tablet_service.cc:1511] Processing CreateTablet for tablet 5d09b12ff1334907b9032e5b613a642e (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:27.671273  4755 tablet_service.cc:1511] Processing CreateTablet for tablet 20fcae796dda4168807069fba8c1974a (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:27.673014  4758 tablet_service.cc:1511] Processing CreateTablet for tablet a813d5c1b5e5457dbf4abaa598e36f37 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:27.673221  4754 tablet_service.cc:1511] Processing CreateTablet for tablet b07adfc14da14b3faced311b3a96e2a4 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:27.675189  4757 tablet_service.cc:1511] Processing CreateTablet for tablet 11e3af0616a047368a0ca6f6d13a72e9 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:27.676783  4605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b07adfc14da14b3faced311b3a96e2a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.678640  4756 tablet_service.cc:1511] Processing CreateTablet for tablet e59d5484fea940eab3ee94cc6dadfea5 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:27.681180  4607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59d5484fea940eab3ee94cc6dadfea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.684808  4609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a813d5c1b5e5457dbf4abaa598e36f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.688378  4763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f228c374a7634467987eaac91c718aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.689189  4760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd465e5c5472416a917ba1a4bc550a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.697722  4689 tablet_service.cc:1511] Processing CreateTablet for tablet f228c374a7634467987eaac91c718aac (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:27.698614  4759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d09b12ff1334907b9032e5b613a642e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.699471  4689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f228c374a7634467987eaac91c718aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.700133  4762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c2e3df5f3c45bf886fff877b2c8ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.700950  4756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59d5484fea940eab3ee94cc6dadfea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.701161  4688 tablet_service.cc:1511] Processing CreateTablet for tablet 82c2e3df5f3c45bf886fff877b2c8ca4 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:27.701655  4757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11e3af0616a047368a0ca6f6d13a72e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.702313  4755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fcae796dda4168807069fba8c1974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.702502  4688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c2e3df5f3c45bf886fff877b2c8ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.705158  4687 tablet_service.cc:1511] Processing CreateTablet for tablet 2529f93211584032b0ca59e2beddd84a (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:27.706496  4687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2529f93211584032b0ca59e2beddd84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.706650  4761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2529f93211584032b0ca59e2beddd84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.707032  4813 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:27.707554  4813 ts_tablet_manager.cc:1403] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.018s
I20260709 13:14:27.708364  4754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b07adfc14da14b3faced311b3a96e2a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.709229  4686 tablet_service.cc:1511] Processing CreateTablet for tablet dd465e5c5472416a917ba1a4bc550a44 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:27.710529  4686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd465e5c5472416a917ba1a4bc550a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.710738  4813 raft_consensus.cc:359] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.711475  4813 raft_consensus.cc:385] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.711786  4813 raft_consensus.cc:740] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.712558  4813 consensus_queue.cc:260] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.713182  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 5d09b12ff1334907b9032e5b613a642e (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:27.714445  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d09b12ff1334907b9032e5b613a642e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.717517  4684 tablet_service.cc:1511] Processing CreateTablet for tablet a813d5c1b5e5457dbf4abaa598e36f37 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:27.720999  4758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a813d5c1b5e5457dbf4abaa598e36f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.720758  4683 tablet_service.cc:1511] Processing CreateTablet for tablet 11e3af0616a047368a0ca6f6d13a72e9 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:27.723801  4682 tablet_service.cc:1511] Processing CreateTablet for tablet e59d5484fea940eab3ee94cc6dadfea5 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:27.725979  4684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a813d5c1b5e5457dbf4abaa598e36f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.727154  4682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59d5484fea940eab3ee94cc6dadfea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.728081  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11e3af0616a047368a0ca6f6d13a72e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.730356  4681 tablet_service.cc:1511] Processing CreateTablet for tablet 20fcae796dda4168807069fba8c1974a (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:27.732640  4681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fcae796dda4168807069fba8c1974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.745944  4680 tablet_service.cc:1511] Processing CreateTablet for tablet b07adfc14da14b3faced311b3a96e2a4 (DEFAULT_TABLE table=test-workload [id=f01fd4d5fc3848699881ea010d7c65f6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:27.747853  4680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b07adfc14da14b3faced311b3a96e2a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:27.759656  4813 ts_tablet_manager.cc:1434] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.041s
I20260709 13:14:27.760658  4649 heartbeater.cc:499] Master 127.27.63.126:39157 was elected leader, sending a full tablet report...
I20260709 13:14:27.761044  4813 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:27.767724  4813 tablet_bootstrap.cc:654] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:27.800249  4650 tablet.cc:2406] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:14:27.845194  4818 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:27.854568  4813 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:27.855278  4813 ts_tablet_manager.cc:1403] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.095s	user 0.038s	sys 0.038s
I20260709 13:14:27.858147  4813 raft_consensus.cc:359] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.858920  4813 raft_consensus.cc:385] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.859278  4813 raft_consensus.cc:740] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.860105  4813 consensus_queue.cc:260] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.862453  4813 ts_tablet_manager.cc:1434] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:27.863535  4813 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:27.869773  4813 tablet_bootstrap.cc:654] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.884631  4815 raft_consensus.cc:493] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.885207  4815 raft_consensus.cc:515] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.888792  4817 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:27.887899  4815 leader_election.cc:290] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:27.901397  4817 tablet_bootstrap.cc:654] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.902894  4818 tablet_bootstrap.cc:654] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.912101  4813 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:27.912660  4813 ts_tablet_manager.cc:1403] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.001s
I20260709 13:14:27.913604  4818 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:27.914180  4818 ts_tablet_manager.cc:1403] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.069s	user 0.007s	sys 0.005s
I20260709 13:14:27.915724  4813 raft_consensus.cc:359] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.916517  4813 raft_consensus.cc:385] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.916810  4813 raft_consensus.cc:740] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.917497  4818 raft_consensus.cc:359] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.917692  4813 consensus_queue.cc:260] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.918324  4818 raft_consensus.cc:385] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.918648  4818 raft_consensus.cc:740] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.919557  4818 consensus_queue.cc:260] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.921034  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c2e3df5f3c45bf886fff877b2c8ca4" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:27.922514  4813 ts_tablet_manager.cc:1434] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:14:27.923580  4813 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:27.927233  4817 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:27.927903  4817 ts_tablet_manager.cc:1403] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.006s
I20260709 13:14:27.928081  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c2e3df5f3c45bf886fff877b2c8ca4" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:27.929426  4724 heartbeater.cc:499] Master 127.27.63.126:39157 was elected leader, sending a full tablet report...
W20260709 13:14:27.929821  4587 leader_election.cc:343] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.930202  4818 ts_tablet_manager.cc:1434] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:14:27.931236  4818 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:27.931734  4817 raft_consensus.cc:359] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.932662  4817 raft_consensus.cc:385] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.932942  4813 tablet_bootstrap.cc:654] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.933080  4817 raft_consensus.cc:740] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
W20260709 13:14:27.935158  4587 leader_election.cc:343] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.935703  4587 leader_election.cc:304] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c87819469991457784c7cf663512487e; no voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:27.937003  4815 raft_consensus.cc:2749] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:27.938264  4818 tablet_bootstrap.cc:654] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.942210  4815 raft_consensus.cc:493] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.942798  4815 raft_consensus.cc:515] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.945359  4815 leader_election.cc:290] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:27.947345  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:27.934298  4817 consensus_queue.cc:260] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.948714  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
W20260709 13:14:27.950654  4587 leader_election.cc:343] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:27.951625  4587 leader_election.cc:343] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.952394  4587 leader_election.cc:304] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c87819469991457784c7cf663512487e; no voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:27.953608  4815 raft_consensus.cc:2749] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:27.959916  4798 heartbeater.cc:499] Master 127.27.63.126:39157 was elected leader, sending a full tablet report...
I20260709 13:14:27.960848  4817 ts_tablet_manager.cc:1434] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.004s
I20260709 13:14:27.961918  4817 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:27.974582  4813 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:27.975195  4813 ts_tablet_manager.cc:1403] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.052s	user 0.008s	sys 0.036s
I20260709 13:14:27.975857  4818 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:27.976492  4818 ts_tablet_manager.cc:1403] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.025s
I20260709 13:14:27.976759  4817 tablet_bootstrap.cc:654] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.978312  4813 raft_consensus.cc:359] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.979152  4813 raft_consensus.cc:385] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.979470  4813 raft_consensus.cc:740] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.979614  4818 raft_consensus.cc:359] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:27.980412  4818 raft_consensus.cc:385] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.980680  4818 raft_consensus.cc:740] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.980640  4813 consensus_queue.cc:260] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:27.981894  4818 consensus_queue.cc:260] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:27.986676  4818 ts_tablet_manager.cc:1434] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:14:27.987782  4818 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:27.992385  4818 tablet_bootstrap.cc:654] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.995672  4813 ts_tablet_manager.cc:1434] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:14:28.001364  4818 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:28.001925  4813 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:28.001925  4818 ts_tablet_manager.cc:1403] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.008s
I20260709 13:14:28.005316  4818 raft_consensus.cc:359] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.006083  4818 raft_consensus.cc:385] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.006376  4818 raft_consensus.cc:740] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.007269  4818 consensus_queue.cc:260] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.009326  4817 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.009625  4813 tablet_bootstrap.cc:654] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.009810  4818 ts_tablet_manager.cc:1434] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:28.009877  4817 ts_tablet_manager.cc:1403] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.048s	user 0.021s	sys 0.016s
I20260709 13:14:28.011126  4818 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:28.013955  4817 raft_consensus.cc:359] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.015035  4817 raft_consensus.cc:385] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.015488  4817 raft_consensus.cc:740] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.016778  4817 consensus_queue.cc:260] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.018317  4818 tablet_bootstrap.cc:654] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.019644  4813 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:28.020474  4827 raft_consensus.cc:493] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.021092  4827 raft_consensus.cc:515] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.022013  4817 ts_tablet_manager.cc:1434] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:14:28.022475  4813 ts_tablet_manager.cc:1403] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 13:14:28.023032  4817 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:28.025565  4813 raft_consensus.cc:359] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:28.026278  4813 raft_consensus.cc:385] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.026584  4813 raft_consensus.cc:740] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.027413  4813 consensus_queue.cc:260] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:28.029695  4813 ts_tablet_manager.cc:1434] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:14:28.030730  4813 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:28.031430  4817 tablet_bootstrap.cc:654] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.032322  4824 raft_consensus.cc:493] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.032824  4824 raft_consensus.cc:515] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.037222  4813 tablet_bootstrap.cc:654] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.041023  4824 leader_election.cc:290] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.056188  4827 leader_election.cc:290] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:28.115331  4827 raft_consensus.cc:493] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.116071  4827 raft_consensus.cc:515] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
W20260709 13:14:28.120793  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:28.126302  4827 leader_election.cc:290] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:28.167572  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
W20260709 13:14:28.169651  4663 leader_election.cc:343] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.172364  4818 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:28.172951  4818 ts_tablet_manager.cc:1403] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.162s	user 0.049s	sys 0.084s
I20260709 13:14:28.173350  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.175539  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.175997  4818 raft_consensus.cc:359] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.178399  4818 raft_consensus.cc:385] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.178532  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.178725  4818 raft_consensus.cc:740] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.178913  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:28.179195  4773 raft_consensus.cc:2468] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.179687  4813 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:28.179508  4818 consensus_queue.cc:260] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.180243  4813 ts_tablet_manager.cc:1403] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.150s	user 0.040s	sys 0.104s
I20260709 13:14:28.180670  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:28.181272  4699 raft_consensus.cc:2468] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 0.
I20260709 13:14:28.182138  4818 ts_tablet_manager.cc:1434] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
W20260709 13:14:28.183133  4736 leader_election.cc:343] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.183274  4813 raft_consensus.cc:359] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
W20260709 13:14:28.177330  4737 leader_election.cc:343] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.184238  4813 raft_consensus.cc:385] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.184620  4813 raft_consensus.cc:740] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
W20260709 13:14:28.184706  4737 leader_election.cc:343] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.185145  4736 leader_election.cc:304] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: c87819469991457784c7cf663512487e
I20260709 13:14:28.185570  4737 leader_election.cc:304] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e
I20260709 13:14:28.185654  4813 consensus_queue.cc:260] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.186681  4815 raft_consensus.cc:493] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.187217  4815 raft_consensus.cc:515] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:28.190050  4815 leader_election.cc:290] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:28.186028  4827 raft_consensus.cc:2804] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.191809  4827 raft_consensus.cc:493] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.192353  4827 raft_consensus.cc:3060] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.197479  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:28.184559  4662 leader_election.cc:304] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: c87819469991457784c7cf663512487e
I20260709 13:14:28.198987  4824 raft_consensus.cc:2804] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.198885  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.199450  4824 raft_consensus.cc:493] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.199802  4824 raft_consensus.cc:3060] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.198180  4699 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87819469991457784c7cf663512487e in term 0.
I20260709 13:14:28.196498  4839 raft_consensus.cc:2749] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:28.202127  4587 leader_election.cc:304] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
W20260709 13:14:28.203320  4587 leader_election.cc:343] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.204509  4815 raft_consensus.cc:2804] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.206096  4815 raft_consensus.cc:493] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.206678  4815 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.183192  4818 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:28.207952  4824 raft_consensus.cc:515] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.211127  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.211444  4817 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.211968  4817 ts_tablet_manager.cc:1403] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.189s	user 0.054s	sys 0.093s
W20260709 13:14:28.212620  4663 leader_election.cc:343] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.214709  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.215111  4817 raft_consensus.cc:359] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.215479  4824 leader_election.cc:290] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.215883  4817 raft_consensus.cc:385] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.216337  4817 raft_consensus.cc:740] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.217098  4817 consensus_queue.cc:260] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.222530  4818 tablet_bootstrap.cc:654] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.222332  4827 raft_consensus.cc:515] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.224021  4773 raft_consensus.cc:2393] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 86c683428f324c35a94717ce212f7276 in current term 1: Already voted for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in this term.
I20260709 13:14:28.226758  4817 ts_tablet_manager.cc:1434] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:14:28.227418  4813 ts_tablet_manager.cc:1434] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.047s	user 0.013s	sys 0.030s
I20260709 13:14:28.228464  4813 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:28.228217  4662 leader_election.cc:304] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86c683428f324c35a94717ce212f7276; no voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:28.234237  4824 raft_consensus.cc:2749] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:28.234915  4813 tablet_bootstrap.cc:654] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.237142  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.239703  4817 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
W20260709 13:14:28.240720  4737 leader_election.cc:343] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.243619  4840 raft_consensus.cc:493] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.243791  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:28.244241  4840 raft_consensus.cc:515] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.246237  4813 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:28.215777  4815 raft_consensus.cc:515] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:28.246493  4840 leader_election.cc:290] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.246773  4813 ts_tablet_manager.cc:1403] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.009s
I20260709 13:14:28.248066  4817 tablet_bootstrap.cc:654] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.246284  4827 leader_election.cc:290] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:28.248854  4815 leader_election.cc:290] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 election: Requested vote from peers fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:28.249960  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.250188  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:28.250095  4813 raft_consensus.cc:359] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.250909  4813 raft_consensus.cc:385] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.251219  4813 raft_consensus.cc:740] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.252189  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.251938  4813 consensus_queue.cc:260] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
W20260709 13:14:28.252732  4587 leader_election.cc:343] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:28.253415  4587 leader_election.cc:343] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:28.254027  4587 leader_election.cc:343] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.254146  4817 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.254204  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:28.254434  4587 leader_election.cc:304] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c87819469991457784c7cf663512487e; no voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:28.254985  4817 ts_tablet_manager.cc:1403] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.005s
I20260709 13:14:28.255110  4698 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.244758  4699 raft_consensus.cc:2393] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in current term 1: Already voted for candidate 86c683428f324c35a94717ce212f7276 in this term.
I20260709 13:14:28.256387  4736 leader_election.cc:304] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e
I20260709 13:14:28.257190  4827 raft_consensus.cc:2749] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:28.259204  4815 raft_consensus.cc:2749] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:28.259404  4818 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:28.260100  4818 ts_tablet_manager.cc:1403] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.011s
I20260709 13:14:28.260174  4817 raft_consensus.cc:359] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.260941  4817 raft_consensus.cc:385] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.261229  4817 raft_consensus.cc:740] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.262363  4813 ts_tablet_manager.cc:1434] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:14:28.262108  4817 consensus_queue.cc:260] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.262661  4698 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87819469991457784c7cf663512487e in term 1.
I20260709 13:14:28.264014  4587 leader_election.cc:304] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:28.265273  4813 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:28.264191  4824 raft_consensus.cc:493] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.265280  4815 raft_consensus.cc:2804] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.265828  4824 raft_consensus.cc:515] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.266275  4815 raft_consensus.cc:697] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 1 LEADER]: Becoming Leader. State: Replica: c87819469991457784c7cf663512487e, State: Running, Role: LEADER
I20260709 13:14:28.269600  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.270593  4818 raft_consensus.cc:359] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
W20260709 13:14:28.271088  4662 leader_election.cc:343] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.271416  4818 raft_consensus.cc:385] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.271831  4818 raft_consensus.cc:740] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.271878  4824 leader_election.cc:290] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.264372  4817 ts_tablet_manager.cc:1434] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:14:28.267146  4815 consensus_queue.cc:237] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:28.272061  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.273015  4818 consensus_queue.cc:260] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.273813  4624 raft_consensus.cc:2468] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.272298  4813 tablet_bootstrap.cc:654] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.275599  4818 ts_tablet_manager.cc:1434] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:14:28.276067  4663 leader_election.cc:304] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:28.276703  4818 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:28.278563  4824 raft_consensus.cc:2804] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.278954  4824 raft_consensus.cc:493] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.279313  4824 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.292973  4817 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:28.294466  4818 tablet_bootstrap.cc:654] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.296327  4824 raft_consensus.cc:515] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.299503  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.300089  4624 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.300241  4813 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:28.300912  4813 ts_tablet_manager.cc:1403] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.036s	user 0.005s	sys 0.027s
I20260709 13:14:28.303689  4813 raft_consensus.cc:359] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.304432  4813 raft_consensus.cc:385] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.304764  4813 raft_consensus.cc:740] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.305482  4813 consensus_queue.cc:260] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.305678  4519 catalog_manager.cc:5697] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e reported cstate change: term changed from 0 to 1, leader changed from <none> to c87819469991457784c7cf663512487e (127.27.63.65). New cstate: current_term: 1 leader_uuid: "c87819469991457784c7cf663512487e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.307231  4813 ts_tablet_manager.cc:1434] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:28.308029  4624 raft_consensus.cc:2468] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.308318  4813 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:28.308506  4818 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:28.309072  4818 ts_tablet_manager.cc:1403] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.000s
I20260709 13:14:28.309424  4663 leader_election.cc:304] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.310384  4846 raft_consensus.cc:2804] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.312026  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.312568  4818 raft_consensus.cc:359] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.313458  4818 raft_consensus.cc:385] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:14:28.313522  4662 leader_election.cc:343] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.313935  4818 raft_consensus.cc:740] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.315114  4813 tablet_bootstrap.cc:654] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.315147  4818 consensus_queue.cc:260] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.318985  4824 leader_election.cc:290] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.322508  4817 tablet_bootstrap.cc:654] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.327920  4846 raft_consensus.cc:697] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:28.329159  4846 consensus_queue.cc:237] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.332391  4848 raft_consensus.cc:493] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.333055  4848 raft_consensus.cc:515] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.336714  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.337359  4624 raft_consensus.cc:2468] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.338828  4663 leader_election.cc:304] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.339713  4824 raft_consensus.cc:2804] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.340235  4824 raft_consensus.cc:493] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.340677  4824 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.343650  4813 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:28.343779  4848 leader_election.cc:290] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.344007  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
W20260709 13:14:28.345582  4662 leader_election.cc:343] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.346530  4813 ts_tablet_manager.cc:1403] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.018s
I20260709 13:14:28.346592  4818 ts_tablet_manager.cc:1434] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.006s
I20260709 13:14:28.348140  4818 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:28.349670  4813 raft_consensus.cc:359] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.351392  4813 raft_consensus.cc:385] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.351728  4813 raft_consensus.cc:740] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.352458  4813 consensus_queue.cc:260] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.354624  4813 ts_tablet_manager.cc:1434] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:14:28.355607  4813 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:28.354858  4519 catalog_manager.cc:5697] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.364233  4818 tablet_bootstrap.cc:654] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.364411  4813 tablet_bootstrap.cc:654] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.364384  4824 raft_consensus.cc:515] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.369804  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.370499  4624 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.383137  4624 raft_consensus.cc:2468] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.391407  4813 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: No bootstrap required, opened a new log
I20260709 13:14:28.392650  4813 ts_tablet_manager.cc:1403] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.015s
I20260709 13:14:28.386174  4817 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.394716  4663 leader_election.cc:304] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.388471  4824 leader_election.cc:290] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.391407  4818 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:28.395519  4824 raft_consensus.cc:2804] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.395344  4813 raft_consensus.cc:359] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.396080  4813 raft_consensus.cc:385] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.394838  4817 ts_tablet_manager.cc:1403] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.102s	user 0.011s	sys 0.015s
I20260709 13:14:28.387908  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.396522  4813 raft_consensus.cc:740] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.396031  4824 raft_consensus.cc:697] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
W20260709 13:14:28.397539  4662 leader_election.cc:343] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:28.397264  4813 consensus_queue.cc:260] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.397675  4824 consensus_queue.cc:237] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.399338  4813 ts_tablet_manager.cc:1434] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:14:28.402433  4817 raft_consensus.cc:359] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:28.403501  4818 ts_tablet_manager.cc:1403] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.056s	user 0.024s	sys 0.006s
I20260709 13:14:28.403891  4817 raft_consensus.cc:385] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.404327  4817 raft_consensus.cc:740] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.405135  4817 consensus_queue.cc:260] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:28.406302  4519 catalog_manager.cc:5697] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.407840  4817 ts_tablet_manager.cc:1434] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:14:28.408977  4817 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:28.409688  4818 raft_consensus.cc:359] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.410983  4818 raft_consensus.cc:385] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.411296  4818 raft_consensus.cc:740] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.412177  4818 consensus_queue.cc:260] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.414773  4818 ts_tablet_manager.cc:1434] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.001s
I20260709 13:14:28.415807  4818 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:28.416455  4817 tablet_bootstrap.cc:654] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.423247  4818 tablet_bootstrap.cc:654] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.424803  4817 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.425352  4817 ts_tablet_manager.cc:1403] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:14:28.428256  4818 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:28.428772  4818 ts_tablet_manager.cc:1403] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.005s
I20260709 13:14:28.428690  4817 raft_consensus.cc:359] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.429558  4817 raft_consensus.cc:385] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.429865  4817 raft_consensus.cc:740] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.430613  4817 consensus_queue.cc:260] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.431507  4818 raft_consensus.cc:359] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.432221  4818 raft_consensus.cc:385] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.432482  4818 raft_consensus.cc:740] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.432848  4817 ts_tablet_manager.cc:1434] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:28.433187  4818 consensus_queue.cc:260] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.433858  4817 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:28.435123  4818 ts_tablet_manager.cc:1434] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:28.436195  4818 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:28.440692  4817 tablet_bootstrap.cc:654] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.441411  4818 tablet_bootstrap.cc:654] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.449141  4817 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.449676  4817 ts_tablet_manager.cc:1403] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 13:14:28.449847  4818 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:28.450356  4818 ts_tablet_manager.cc:1403] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:14:28.452257  4817 raft_consensus.cc:359] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.452400  4818 raft_consensus.cc:359] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.452798  4817 raft_consensus.cc:385] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.453020  4818 raft_consensus.cc:385] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.453155  4817 raft_consensus.cc:740] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.453367  4818 raft_consensus.cc:740] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.453949  4817 consensus_queue.cc:260] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.454056  4818 consensus_queue.cc:260] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.455976  4818 ts_tablet_manager.cc:1434] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:28.456118  4817 ts_tablet_manager.cc:1434] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:14:28.456616  4827 raft_consensus.cc:493] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.457199  4817 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:28.457108  4827 raft_consensus.cc:515] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.458087  4818 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276: Bootstrap starting.
I20260709 13:14:28.459247  4827 leader_election.cc:290] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:28.463881  4818 tablet_bootstrap.cc:654] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.463881  4817 tablet_bootstrap.cc:654] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.465260  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.466073  4624 raft_consensus.cc:2393] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in current term 1: Already voted for candidate 86c683428f324c35a94717ce212f7276 in this term.
I20260709 13:14:28.467974  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:28.470391  4736 leader_election.cc:304] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e
I20260709 13:14:28.471302  4818 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276: No bootstrap required, opened a new log
I20260709 13:14:28.464082  4815 raft_consensus.cc:493] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.471793  4818 ts_tablet_manager.cc:1403] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:14:28.472141  4815 raft_consensus.cc:515] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.474237  4815 leader_election.cc:290] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.474436  4818 raft_consensus.cc:359] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.475153  4818 raft_consensus.cc:385] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.475232  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.475449  4818 raft_consensus.cc:740] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.475656  4827 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.475908  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:28.476603  4698 raft_consensus.cc:2468] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87819469991457784c7cf663512487e in term 0.
I20260709 13:14:28.476461  4818 consensus_queue.cc:260] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.475924  4773 raft_consensus.cc:2468] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87819469991457784c7cf663512487e in term 0.
I20260709 13:14:28.478477  4587 leader_election.cc:304] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:28.478811  4818 ts_tablet_manager.cc:1434] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:28.481886  4817 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.482371  4817 ts_tablet_manager.cc:1403] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.006s
I20260709 13:14:28.483170  4827 raft_consensus.cc:2749] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:28.483955  4815 raft_consensus.cc:2804] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.484372  4815 raft_consensus.cc:493] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.484735  4815 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.484555  4817 raft_consensus.cc:359] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.485262  4817 raft_consensus.cc:385] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.485505  4817 raft_consensus.cc:740] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.486191  4817 consensus_queue.cc:260] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.487680  4817 ts_tablet_manager.cc:1434] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:28.488471  4817 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:28.491529  4815 raft_consensus.cc:515] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.492457  4817 tablet_bootstrap.cc:654] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.493719  4815 leader_election.cc:290] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 election: Requested vote from peers 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.494762  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.494874  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:28.495607  4773 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.495661  4698 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.497511  4817 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.497997  4817 ts_tablet_manager.cc:1403] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:14:28.500735  4817 raft_consensus.cc:359] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.501386  4817 raft_consensus.cc:385] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.501695  4817 raft_consensus.cc:740] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.502303  4817 consensus_queue.cc:260] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.502924  4773 raft_consensus.cc:2468] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87819469991457784c7cf663512487e in term 1.
I20260709 13:14:28.503196  4698 raft_consensus.cc:2468] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87819469991457784c7cf663512487e in term 1.
I20260709 13:14:28.504359  4817 ts_tablet_manager.cc:1434] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:14:28.504434  4587 leader_election.cc:304] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.505295  4815 raft_consensus.cc:2804] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.505425  4817 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Bootstrap starting.
I20260709 13:14:28.505800  4815 raft_consensus.cc:697] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 1 LEADER]: Becoming Leader. State: Replica: c87819469991457784c7cf663512487e, State: Running, Role: LEADER
I20260709 13:14:28.506680  4815 consensus_queue.cc:237] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.511432  4817 tablet_bootstrap.cc:654] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.515161  4521 catalog_manager.cc:5697] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e reported cstate change: term changed from 0 to 1, leader changed from <none> to c87819469991457784c7cf663512487e (127.27.63.65). New cstate: current_term: 1 leader_uuid: "c87819469991457784c7cf663512487e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.515820  4817 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: No bootstrap required, opened a new log
I20260709 13:14:28.516369  4817 ts_tablet_manager.cc:1403] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:14:28.517495  4824 raft_consensus.cc:493] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.517927  4824 raft_consensus.cc:515] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.518919  4817 raft_consensus.cc:359] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.519562  4817 raft_consensus.cc:385] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.519819  4817 raft_consensus.cc:740] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.520125  4824 leader_election.cc:290] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.520478  4817 consensus_queue.cc:260] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.521495  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59d5484fea940eab3ee94cc6dadfea5" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.521899  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59d5484fea940eab3ee94cc6dadfea5" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.522198  4773 raft_consensus.cc:2468] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.522531  4817 ts_tablet_manager.cc:1434] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:28.523524  4662 leader_election.cc:304] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:28.524252  4824 raft_consensus.cc:2804] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.524662  4824 raft_consensus.cc:493] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.522555  4624 raft_consensus.cc:2468] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.525228  4824 raft_consensus.cc:3060] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.532361  4824 raft_consensus.cc:515] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.533517  4846 raft_consensus.cc:493] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.534022  4846 raft_consensus.cc:515] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.534360  4824 leader_election.cc:290] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.535470  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59d5484fea940eab3ee94cc6dadfea5" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.536108  4846 leader_election.cc:290] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.536937  4624 raft_consensus.cc:3060] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.536777  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59d5484fea940eab3ee94cc6dadfea5" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.536835  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a813d5c1b5e5457dbf4abaa598e36f37" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.537577  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a813d5c1b5e5457dbf4abaa598e36f37" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.537786  4773 raft_consensus.cc:3060] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.537868  4623 raft_consensus.cc:2468] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.538275  4772 raft_consensus.cc:2468] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.539429  4662 leader_election.cc:304] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:28.540520  4846 raft_consensus.cc:2804] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.540874  4846 raft_consensus.cc:493] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.541221  4846 raft_consensus.cc:3060] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.543874  4773 raft_consensus.cc:2468] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.544698  4624 raft_consensus.cc:2468] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.544912  4662 leader_election.cc:304] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:28.545845  4824 raft_consensus.cc:2804] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.546245  4824 raft_consensus.cc:697] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:28.546985  4824 consensus_queue.cc:237] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.547814  4846 raft_consensus.cc:515] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.550515  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a813d5c1b5e5457dbf4abaa598e36f37" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.550559  4846 leader_election.cc:290] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.550828  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a813d5c1b5e5457dbf4abaa598e36f37" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.551254  4624 raft_consensus.cc:3060] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.551491  4773 raft_consensus.cc:3060] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.556011  4519 catalog_manager.cc:5697] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.558151  4624 raft_consensus.cc:2468] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.558151  4773 raft_consensus.cc:2468] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.559350  4663 leader_election.cc:304] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.560482  4846 raft_consensus.cc:2804] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.560943  4846 raft_consensus.cc:697] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:28.561636  4846 consensus_queue.cc:237] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.569310  4519 catalog_manager.cc:5697] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.572739  4846 raft_consensus.cc:493] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.573305  4846 raft_consensus.cc:515] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.575294  4846 leader_election.cc:290] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.575943  4827 raft_consensus.cc:493] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.576637  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.576494  4827 raft_consensus.cc:515] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.576862  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.577275  4773 raft_consensus.cc:2468] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.577364  4624 raft_consensus.cc:2468] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.578503  4662 leader_election.cc:304] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:28.578642  4827 leader_election.cc:290] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:28.580111  4846 raft_consensus.cc:2804] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.579968  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.580633  4846 raft_consensus.cc:493] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.580490  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:28.580981  4624 raft_consensus.cc:2393] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in current term 1: Already voted for candidate 86c683428f324c35a94717ce212f7276 in this term.
I20260709 13:14:28.581074  4846 raft_consensus.cc:3060] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.582283  4736 leader_election.cc:304] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e
I20260709 13:14:28.582929  4827 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.586476  4846 raft_consensus.cc:515] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.587656  4827 raft_consensus.cc:2749] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:28.588680  4846 leader_election.cc:290] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.589102  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.589241  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.589624  4624 raft_consensus.cc:3060] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.589785  4773 raft_consensus.cc:3060] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.594475  4624 raft_consensus.cc:2468] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.594758  4773 raft_consensus.cc:2468] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.595587  4663 leader_election.cc:304] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.596421  4846 raft_consensus.cc:2804] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.596900  4846 raft_consensus.cc:697] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:28.597675  4846 consensus_queue.cc:237] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.599442  4815 raft_consensus.cc:493] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.599922  4815 raft_consensus.cc:515] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.602429  4815 leader_election.cc:290] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.603160  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:28.603974  4698 raft_consensus.cc:2393] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c87819469991457784c7cf663512487e in current term 1: Already voted for candidate 86c683428f324c35a94717ce212f7276 in this term.
I20260709 13:14:28.603991  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "c87819469991457784c7cf663512487e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.604830  4773 raft_consensus.cc:2393] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c87819469991457784c7cf663512487e in current term 1: Already voted for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in this term.
I20260709 13:14:28.605764  4587 leader_election.cc:304] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c87819469991457784c7cf663512487e; no voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:28.606596  4815 raft_consensus.cc:3060] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.606153  4519 catalog_manager.cc:5697] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.612757  4815 raft_consensus.cc:2749] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:28.642014  4846 raft_consensus.cc:493] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.642607  4846 raft_consensus.cc:515] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.644908  4846 leader_election.cc:290] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.645287  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c2e3df5f3c45bf886fff877b2c8ca4" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.645512  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c2e3df5f3c45bf886fff877b2c8ca4" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.646016  4624 raft_consensus.cc:2468] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.646157  4773 raft_consensus.cc:2468] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.647138  4663 leader_election.cc:304] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.648063  4846 raft_consensus.cc:2804] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.648417  4846 raft_consensus.cc:493] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.648720  4846 raft_consensus.cc:3060] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.654196  4846 raft_consensus.cc:515] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.655768  4846 leader_election.cc:290] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.656543  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c2e3df5f3c45bf886fff877b2c8ca4" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.656738  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c2e3df5f3c45bf886fff877b2c8ca4" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.657189  4624 raft_consensus.cc:3060] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.657239  4773 raft_consensus.cc:3060] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.664001  4773 raft_consensus.cc:2468] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.664001  4624 raft_consensus.cc:2468] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.665472  4663 leader_election.cc:304] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.666181  4846 raft_consensus.cc:2804] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.666621  4846 raft_consensus.cc:697] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:28.667331  4846 consensus_queue.cc:237] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.675791  4519 catalog_manager.cc:5697] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.726133  4846 consensus_queue.cc:1048] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.737478  4824 consensus_queue.cc:1048] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.763345  4773 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.769482  4815 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.779256  4850 raft_consensus.cc:493] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.779760  4850 raft_consensus.cc:515] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.781286  4815 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:28.783062  4850 leader_election.cc:290] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.783761  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:28.784462  4624 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.785665  4663 leader_election.cc:304] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.786238  4850 consensus_queue.cc:1048] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.786522  4824 raft_consensus.cc:2804] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:28.786926  4824 raft_consensus.cc:493] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.787251  4824 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.796679  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
I20260709 13:14:28.797842  4773 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 0.
I20260709 13:14:28.795703  4824 raft_consensus.cc:515] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.800292  4824 leader_election.cc:290] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:28.801970  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:28.802596  4624 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.805930  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:28.806581  4772 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.808238  4850 consensus_queue.cc:1048] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.809964  4624 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.813740  4663 leader_election.cc:304] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:28.815766  4850 raft_consensus.cc:2804] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.816272  4850 raft_consensus.cc:697] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:28.816808  4772 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 1.
I20260709 13:14:28.817142  4850 consensus_queue.cc:237] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:28.832613  4522 catalog_manager.cc:5697] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.920462  4815 consensus_queue.cc:1048] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.930758  4878 consensus_queue.cc:1048] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.982151  4850 consensus_queue.cc:1048] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.991803  4848 consensus_queue.cc:1048] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:29.085827  4848 consensus_queue.cc:1048] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:29.096755  4824 consensus_queue.cc:1048] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:29.100864  4850 consensus_queue.cc:1048] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:29.114295  4848 consensus_queue.cc:1048] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:29.128973  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:29.173563  4848 consensus_queue.cc:1048] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:29.183760  4824 consensus_queue.cc:1048] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:29.391557  4824 consensus_queue.cc:1048] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:29.401363  4824 consensus_queue.cc:1048] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.077311  4895 raft_consensus.cc:493] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.077824  4895 raft_consensus.cc:515] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:30.080241  4895 leader_election.cc:290] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:30.080715  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e" is_pre_election: true
I20260709 13:14:30.081175  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:30.081485  4623 raft_consensus.cc:2468] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 1.
I20260709 13:14:30.081831  4698 raft_consensus.cc:2468] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 1.
I20260709 13:14:30.082712  4737 leader_election.cc:304] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:30.083528  4895 raft_consensus.cc:2804] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:30.083798  4895 raft_consensus.cc:493] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.084007  4895 raft_consensus.cc:3060] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:30.088845  4895 raft_consensus.cc:515] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:30.090389  4895 leader_election.cc:290] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:30.091228  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:30.091413  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcae796dda4168807069fba8c1974a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:30.091771  4623 raft_consensus.cc:3060] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:30.092089  4698 raft_consensus.cc:3060] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:30.097101  4623 raft_consensus.cc:2468] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:30.097312  4698 raft_consensus.cc:2468] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:30.098280  4737 leader_election.cc:304] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:30.099040  4895 raft_consensus.cc:2804] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:30.099877  4895 raft_consensus.cc:697] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:30.100742  4895 consensus_queue.cc:237] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:30.108260  4519 catalog_manager.cc:5697] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 0 to 2, leader changed from <none> to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:30.407900 27901 ts_manager.cc:295] Set tserver state for c87819469991457784c7cf663512487e to MAINTENANCE_MODE
I20260709 13:14:30.408712 27901 tablet_server.cc:179] TabletServer@127.27.63.65:0 shutting down...
W20260709 13:14:30.415374  4663 consensus_peers.cc:597] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:30.435063 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:30.435767 27901 tablet_replica.cc:333] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.436408 27901 raft_consensus.cc:2243] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:30.436903 27901 raft_consensus.cc:2272] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.438764 27901 tablet_replica.cc:333] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.439232 27901 raft_consensus.cc:2243] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:30.439703 27901 raft_consensus.cc:2272] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.441684 27901 tablet_replica.cc:333] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.442404 27901 raft_consensus.cc:2243] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:30.442782 27901 raft_consensus.cc:2272] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.444468 27901 tablet_replica.cc:333] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.444960 27901 raft_consensus.cc:2243] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:30.445652 27901 raft_consensus.cc:2272] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.447338 27901 tablet_replica.cc:333] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.447866 27901 raft_consensus.cc:2243] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:30.448280 27901 raft_consensus.cc:2272] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.449891 27901 tablet_replica.cc:333] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.450379 27901 raft_consensus.cc:2243] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:30.450757 27901 raft_consensus.cc:2272] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.452451 27901 tablet_replica.cc:333] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.452932 27901 raft_consensus.cc:2243] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:30.453620 27901 raft_consensus.cc:2272] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.455324 27901 tablet_replica.cc:333] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.455825 27901 raft_consensus.cc:2243] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:30.456230 27901 raft_consensus.cc:2272] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.457876 27901 tablet_replica.cc:333] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.458360 27901 raft_consensus.cc:2243] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:30.458736 27901 raft_consensus.cc:2272] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.460378 27901 tablet_replica.cc:333] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: stopping tablet replica
I20260709 13:14:30.460841 27901 raft_consensus.cc:2243] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:30.461249 27901 raft_consensus.cc:2272] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:30.490278 27901 tablet_server.cc:196] TabletServer@127.27.63.65:0 shutdown complete.
W20260709 13:14:30.508339  4663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111) [suppressed 86 similar messages]
W20260709 13:14:30.511365  4663 consensus_peers.cc:597] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:30.530522  4737 consensus_peers.cc:597] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:30.553800  4698 raft_consensus.cc:1275] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:30.555186  4895 consensus_queue.cc:1048] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:30.556372  4663 consensus_peers.cc:597] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:30.668526  4663 consensus_peers.cc:597] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:30.685437  4663 consensus_peers.cc:597] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:30.723587  4663 consensus_peers.cc:597] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:30.786960  4663 consensus_peers.cc:597] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:31.142225  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:14:31.580184  4848 raft_consensus.cc:493] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c87819469991457784c7cf663512487e)
I20260709 13:14:31.580757  4848 raft_consensus.cc:515] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:31.582417  4848 leader_election.cc:290] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:31.583371  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
W20260709 13:14:31.585809  4663 leader_election.cc:336] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111)
I20260709 13:14:31.586232  4663 leader_election.cc:304] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86c683428f324c35a94717ce212f7276; no voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:31.586884  4848 raft_consensus.cc:2749] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:31.619587  4895 raft_consensus.cc:493] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c87819469991457784c7cf663512487e)
I20260709 13:14:31.620177  4895 raft_consensus.cc:515] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:31.621843  4895 leader_election.cc:290] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:31.622993  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:31.623679  4698 raft_consensus.cc:2468] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 1.
I20260709 13:14:31.624763  4736 leader_election.cc:304] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:31.625456  4895 raft_consensus.cc:2804] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:14:31.625543  4737 leader_election.cc:336] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111)
I20260709 13:14:31.625838  4895 raft_consensus.cc:493] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c87819469991457784c7cf663512487e)
I20260709 13:14:31.626226  4895 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:31.631153  4895 raft_consensus.cc:515] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:31.632701  4895 leader_election.cc:290] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:31.633761  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:31.634388  4698 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:31.636123  4737 leader_election.cc:336] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111)
I20260709 13:14:31.639230  4698 raft_consensus.cc:2468] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:31.640249  4736 leader_election.cc:304] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: c87819469991457784c7cf663512487e
I20260709 13:14:31.640848  4895 raft_consensus.cc:2804] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:31.641817  4895 raft_consensus.cc:697] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:31.642496  4895 consensus_queue.cc:237] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:31.649008  4522 catalog_manager.cc:5697] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 1 to 2, leader changed from c87819469991457784c7cf663512487e (127.27.63.65) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:31.801720  4848 raft_consensus.cc:493] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c87819469991457784c7cf663512487e)
I20260709 13:14:31.802255  4848 raft_consensus.cc:515] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:31.803893  4848 leader_election.cc:290] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:31.805389  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" is_pre_election: true
W20260709 13:14:31.808554  4663 leader_election.cc:336] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111)
I20260709 13:14:31.808974  4663 leader_election.cc:304] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86c683428f324c35a94717ce212f7276; no voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:31.845062  4848 raft_consensus.cc:2749] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:31.942796  4895 raft_consensus.cc:493] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c87819469991457784c7cf663512487e)
I20260709 13:14:31.943805  4895 raft_consensus.cc:515] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:31.946590  4895 leader_election.cc:290] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:31.948525  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276" is_pre_election: true
I20260709 13:14:31.949501  4698 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 1.
W20260709 13:14:31.950863  4737 leader_election.cc:336] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111)
I20260709 13:14:31.951539  4736 leader_election.cc:304] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: c87819469991457784c7cf663512487e
I20260709 13:14:31.952252  4895 raft_consensus.cc:2804] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:31.952608  4895 raft_consensus.cc:493] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c87819469991457784c7cf663512487e)
I20260709 13:14:31.952890  4895 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:31.961855  4895 raft_consensus.cc:515] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:31.963619  4895 leader_election.cc:290] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:31.964813  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:31.965463  4698 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:31.967214  4737 leader_election.cc:336] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111)
I20260709 13:14:31.972270  4698 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:31.973358  4736 leader_election.cc:304] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86c683428f324c35a94717ce212f7276, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: c87819469991457784c7cf663512487e
I20260709 13:14:31.973996  4895 raft_consensus.cc:2804] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:31.974400  4895 raft_consensus.cc:697] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:31.975479  4895 consensus_queue.cc:237] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:31.985877  4522 catalog_manager.cc:5697] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 1 to 2, leader changed from c87819469991457784c7cf663512487e (127.27.63.65) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:32.111454  4698 raft_consensus.cc:1275] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:32.112819  4895 consensus_queue.cc:1048] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:32.129105  4737 consensus_peers.cc:597] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:32.143407  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:14:32.396427  4698 raft_consensus.cc:1275] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:32.397902  4911 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:32.411914  4737 consensus_peers.cc:597] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:32.799661  4737 consensus_peers.cc:597] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:32.836305  4663 consensus_peers.cc:597] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:32.935348  4663 consensus_peers.cc:597] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:33.027904  4663 consensus_peers.cc:597] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:33.144770  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:14:33.160092  4663 consensus_peers.cc:597] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:33.174959  4663 consensus_peers.cc:597] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:33.260895  4663 consensus_peers.cc:597] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:33.550792  4663 consensus_peers.cc:597] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:34.145758  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:14:34.576803  4737 consensus_peers.cc:597] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:34.781329  4737 consensus_peers.cc:597] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:35.146631  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:14:35.291378  4663 consensus_peers.cc:597] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:35.380381  4663 consensus_peers.cc:597] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:35.413533  4737 consensus_peers.cc:597] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:35.512740 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:35.520373  4928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:35.520437  4929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:35.523981  4931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:35.524125 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:35.526074 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:35.526310 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:35.526512 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602875526494 us; error 0 us; skew 500 ppm
I20260709 13:14:35.527071 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:35.529743 27901 webserver.cc:533] Webserver started at http://127.27.63.65:33975/ using document root <none> and password file <none>
I20260709 13:14:35.530309 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:35.530521 27901 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:14:35.534439 27901 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:14:35.538699  4936 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:35.539518 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:14:35.539840 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "c87819469991457784c7cf663512487e"
format_stamp: "Formatted at 2026-07-09 13:14:27 on dist-test-slave-sjgx"
I20260709 13:14:35.540272 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:14:35.548597  4663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111) [suppressed 101 similar messages]
W20260709 13:14:35.550833  4663 consensus_peers.cc:597] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:35.562500 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:35.563880 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:35.565352 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:35.569094  4943 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:14:35.618108 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:14:35.618443 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.050s	user 0.002s	sys 0.000s
I20260709 13:14:35.618773 27901 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:14:35.623992  4943 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.637570  4943 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.638339  4943 tablet_bootstrap.cc:492] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.638880  4943 ts_tablet_manager.cc:1403] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:14:35.641047  4943 raft_consensus.cc:359] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.641597  4943 raft_consensus.cc:740] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.642338  4943 consensus_queue.cc:260] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.644696  4943 ts_tablet_manager.cc:1434] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:14:35.645346  4943 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.656328 27901 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:14:35.656647 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.038s	user 0.038s	sys 0.000s
I20260709 13:14:35.657594  4943 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.658416  4943 tablet_bootstrap.cc:492] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.658959  4943 ts_tablet_manager.cc:1403] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:14:35.661266  4943 raft_consensus.cc:359] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.661891  4943 raft_consensus.cc:740] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.662536  4943 consensus_queue.cc:260] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.664001  4943 ts_tablet_manager.cc:1434] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:35.664743  4943 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.679301  4943 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.680142  4943 tablet_bootstrap.cc:492] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.680920  4943 ts_tablet_manager.cc:1403] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:14:35.683109  4943 raft_consensus.cc:359] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.683667  4943 raft_consensus.cc:740] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.684384  4943 consensus_queue.cc:260] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.687534  4943 ts_tablet_manager.cc:1434] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:35.688335  4943 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.703964  4943 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.704720  4943 tablet_bootstrap.cc:492] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.705240  4943 ts_tablet_manager.cc:1403] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:14:35.707243  4943 raft_consensus.cc:359] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.707744  4943 raft_consensus.cc:740] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.708385  4943 consensus_queue.cc:260] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.709900  4943 ts_tablet_manager.cc:1434] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:35.710594  4943 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.724642 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:35357
I20260709 13:14:35.724754  5005 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:35357 every 8 connection(s)
I20260709 13:14:35.725248  4943 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.726287  4943 tablet_bootstrap.cc:492] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.727063  4943 ts_tablet_manager.cc:1403] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:14:35.729596  4943 raft_consensus.cc:359] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.730345  4943 raft_consensus.cc:740] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.731076  4943 consensus_queue.cc:260] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.733312  4943 ts_tablet_manager.cc:1434] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:35.734210  4943 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.739681 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:35.756791  5007 heartbeater.cc:344] Connected to a master server at 127.27.63.126:39157
I20260709 13:14:35.757354  5007 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:35.758211  4943 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.758353  5007 heartbeater.cc:507] Master 127.27.63.126:39157 requested a full tablet report, sending...
I20260709 13:14:35.759315  4943 tablet_bootstrap.cc:492] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.759935  4943 ts_tablet_manager.cc:1403] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.008s
I20260709 13:14:35.762334  4943 raft_consensus.cc:359] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:35.763031  4943 raft_consensus.cc:740] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.763547  4943 consensus_queue.cc:260] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:35.764537  4519 ts_manager.cc:194] Re-registered known tserver with Master: c87819469991457784c7cf663512487e (127.27.63.65:35357)
I20260709 13:14:35.765113  4943 ts_tablet_manager.cc:1434] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:35.765761  4943 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.771859  4519 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42786
I20260709 13:14:35.775547  5007 heartbeater.cc:499] Master 127.27.63.126:39157 was elected leader, sending a full tablet report...
I20260709 13:14:35.777715  4698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9"
dest_uuid: "86c683428f324c35a94717ce212f7276"
mode: GRACEFUL
new_leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:14:35.778107  4943 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.778508  4698 raft_consensus.cc:606] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Received request to transfer leadership to TS fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:35.779150  4943 tablet_bootstrap.cc:492] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.779867  4943 ts_tablet_manager.cc:1403] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:14:35.781247  4698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e"
dest_uuid: "86c683428f324c35a94717ce212f7276"
mode: GRACEFUL
new_leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:14:35.781751  4698 raft_consensus.cc:606] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Received request to transfer leadership to TS fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:35.782465  4943 raft_consensus.cc:359] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.783164  4943 raft_consensus.cc:740] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.784076  4698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44"
dest_uuid: "86c683428f324c35a94717ce212f7276"
mode: GRACEFUL
new_leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:14:35.783865  4943 consensus_queue.cc:260] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.784621  4698 raft_consensus.cc:606] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Received request to transfer leadership to TS fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:35.785784 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:35.786083  4943 ts_tablet_manager.cc:1434] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:35.786285 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:35.786891  4943 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.787009 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:35.801478  4943 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.802366  4943 tablet_bootstrap.cc:492] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.803040  4943 ts_tablet_manager.cc:1403] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
W20260709 13:14:35.803738  4663 consensus_peers.cc:597] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:35.805019  4943 raft_consensus.cc:359] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.805557  4943 raft_consensus.cc:740] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.806142  4943 consensus_queue.cc:260] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.807648  4943 ts_tablet_manager.cc:1434] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:35.808398  4943 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.823841  4943 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.824962  4943 tablet_bootstrap.cc:492] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.825604  4943 ts_tablet_manager.cc:1403] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:14:35.826951  4979 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:35.829356  4943 raft_consensus.cc:359] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.830202  4943 raft_consensus.cc:740] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.831126  4943 consensus_queue.cc:260] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.833720  4943 ts_tablet_manager.cc:1434] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:14:35.834571  4943 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Bootstrap starting.
I20260709 13:14:35.850687  4943 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.851475  4943 tablet_bootstrap.cc:492] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Bootstrap complete.
I20260709 13:14:35.852085  4943 ts_tablet_manager.cc:1403] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 13:14:35.854204  4943 raft_consensus.cc:359] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.854756  4943 raft_consensus.cc:740] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87819469991457784c7cf663512487e, State: Initialized, Role: FOLLOWER
I20260709 13:14:35.855417  4943 consensus_queue.cc:260] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.857151  4943 ts_tablet_manager.cc:1434] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:35.899550  4848 raft_consensus.cc:993] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276: : Instructing follower fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:35.900101  4848 raft_consensus.cc:1081] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Signalling peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:35.901692  4772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:48414
I20260709 13:14:35.902235  4772 raft_consensus.cc:493] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:35.902563  4772 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:35.909232  4772 raft_consensus.cc:515] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.911257  4772 leader_election.cc:290] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:35.911919  4979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:35.912402  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:35.912667  4979 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:35.913004  4698 raft_consensus.cc:3055] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:35.913337  4698 raft_consensus.cc:740] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:35.914234  4698 consensus_queue.cc:260] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.915675  4698 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:35.920193  4979 raft_consensus.cc:2468] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:35.921429  4737 leader_election.cc:304] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:35.922276  4926 raft_consensus.cc:2804] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:35.923362  4698 raft_consensus.cc:2468] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:35.923522  4926 raft_consensus.cc:697] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:35.924690  4926 consensus_queue.cc:237] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.931749  4927 raft_consensus.cc:993] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276: : Instructing follower fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:35.931365  4519 catalog_manager.cc:5697] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 1 to 2, leader changed from 86c683428f324c35a94717ce212f7276 (127.27.63.66) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:35.932269  4848 raft_consensus.cc:1081] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Signalling peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:35.933723  4772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:48414
I20260709 13:14:35.934172  4772 raft_consensus.cc:493] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:35.934482  4772 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:35.941082  4772 raft_consensus.cc:515] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.943884  4979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:35.943754  4772 leader_election.cc:290] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:35.944566  4979 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:35.944736  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:35.945350  4698 raft_consensus.cc:3055] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:35.945645  4698 raft_consensus.cc:740] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:35.946328  4698 consensus_queue.cc:260] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.947305  4698 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:35.949756  4979 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:35.950763  4737 leader_election.cc:304] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:35.951706  4926 raft_consensus.cc:2804] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:35.952221  4926 raft_consensus.cc:697] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:35.953233  4926 consensus_queue.cc:237] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:35.954034  4698 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:35.960991  4519 catalog_manager.cc:5697] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 1 to 2, leader changed from 86c683428f324c35a94717ce212f7276 (127.27.63.66) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.146857  4979 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:36.147626  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:14:36.236090  4927 raft_consensus.cc:993] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276: : Instructing follower fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:36.236635  4848 raft_consensus.cc:1081] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Signalling peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:36.237958  4772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:48414
I20260709 13:14:36.238427  4772 raft_consensus.cc:493] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:36.238714  4772 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:36.244148  4772 raft_consensus.cc:515] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:36.247120  4979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:36.247712  4979 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:36.248981  4772 leader_election.cc:290] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:36.249329  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:36.249984  4698 raft_consensus.cc:3055] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:36.250339  4698 raft_consensus.cc:740] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:36.251507  4698 consensus_queue.cc:260] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:36.253374  4698 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:36.255859  4979 raft_consensus.cc:2468] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:36.257762  4737 leader_election.cc:304] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:36.259024  5029 raft_consensus.cc:2804] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:36.259625  5029 raft_consensus.cc:697] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:36.261004  5029 consensus_queue.cc:237] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:36.262166  4698 raft_consensus.cc:2468] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:36.270263  4519 catalog_manager.cc:5697] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 1 to 2, leader changed from 86c683428f324c35a94717ce212f7276 (127.27.63.66) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.325153  4698 raft_consensus.cc:1275] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:36.326555  5029 consensus_queue.cc:1048] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:36.337265  4979 raft_consensus.cc:1275] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:36.339061  4926 consensus_queue.cc:1048] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:36.414367  4979 raft_consensus.cc:1275] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:36.415542  4926 consensus_queue.cc:1048] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:36.425505  4698 raft_consensus.cc:1275] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:36.426649  4926 consensus_queue.cc:1048] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:36.767575  4979 raft_consensus.cc:1275] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:36.768990  4926 consensus_queue.cc:1048] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:36.780580  4698 raft_consensus.cc:1275] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:36.783310  5021 consensus_queue.cc:1048] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:36.793824 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:36.810549  4772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
mode: GRACEFUL
new_leader_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:40032
I20260709 13:14:36.811110  4772 raft_consensus.cc:606] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Received request to transfer leadership to TS 86c683428f324c35a94717ce212f7276
I20260709 13:14:36.813450  4772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
mode: GRACEFUL
new_leader_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:40032
I20260709 13:14:36.813884  4772 raft_consensus.cc:606] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Received request to transfer leadership to TS 86c683428f324c35a94717ce212f7276
I20260709 13:14:36.814846 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:14:36.815375 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:36.815970 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:37.027055  4926 raft_consensus.cc:993] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: : Instructing follower 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:37.027644  5021 raft_consensus.cc:1081] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Signalling peer 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:37.029184  4698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a"
dest_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:50164
I20260709 13:14:37.029651  4698 raft_consensus.cc:493] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.029934  4698 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.034886  4698 raft_consensus.cc:515] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:37.036705  4698 leader_election.cc:290] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:37.037549  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:37.038239  4978 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.038487  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2529f93211584032b0ca59e2beddd84a" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:37.039094  4772 raft_consensus.cc:3055] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:37.039467  4772 raft_consensus.cc:740] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:37.040337  4772 consensus_queue.cc:260] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:37.041600  4772 raft_consensus.cc:3060] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.046432  4978 raft_consensus.cc:2468] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:37.047807  4663 leader_election.cc:304] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:37.048595  4848 raft_consensus.cc:2804] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:37.048667  4772 raft_consensus.cc:2468] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:37.049053  4848 raft_consensus.cc:697] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:37.049949  4848 consensus_queue.cc:237] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:37.056514  4519 catalog_manager.cc:5697] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 2 to 3, leader changed from fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67) to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 3 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:37.148695  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:14:37.214120  5029 raft_consensus.cc:993] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6: : Instructing follower 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:37.214655  5021 raft_consensus.cc:1081] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Signalling peer 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:37.216167  4698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e"
dest_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:50164
I20260709 13:14:37.216599  4698 raft_consensus.cc:493] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.216881  4698 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.221531  4698 raft_consensus.cc:515] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:37.222857  4698 leader_election.cc:290] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:37.223658  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:37.223819  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d09b12ff1334907b9032e5b613a642e" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:37.224390  4978 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.224437  4772 raft_consensus.cc:3055] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:37.224879  4772 raft_consensus.cc:740] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:37.225473  4772 consensus_queue.cc:260] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:37.226292  4772 raft_consensus.cc:3060] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.230049  4978 raft_consensus.cc:2468] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:37.231200  4663 leader_election.cc:304] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:37.231446  4772 raft_consensus.cc:2468] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:37.231976  4848 raft_consensus.cc:2804] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:37.232419  4848 raft_consensus.cc:697] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:37.233036  4848 consensus_queue.cc:237] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:37.238444  4519 catalog_manager.cc:5697] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 2 to 3, leader changed from fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67) to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 3 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:37.530838  4772 raft_consensus.cc:1275] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.532032  4848 consensus_queue.cc:1048] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:37.541220  4978 raft_consensus.cc:1275] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.542769  4848 consensus_queue.cc:1048] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:37.622633  4772 raft_consensus.cc:1275] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.623692  4848 consensus_queue.cc:1048] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:37.631233  4978 raft_consensus.cc:1275] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.632990  4848 consensus_queue.cc:1048] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:37.822479 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:37.829394  4698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4"
dest_uuid: "86c683428f324c35a94717ce212f7276"
mode: GRACEFUL
new_leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:14:37.829883  4698 raft_consensus.cc:606] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Received request to transfer leadership to TS fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:37.832199  4698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e59d5484fea940eab3ee94cc6dadfea5"
dest_uuid: "86c683428f324c35a94717ce212f7276"
mode: GRACEFUL
new_leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:14:37.832613  4698 raft_consensus.cc:606] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Received request to transfer leadership to TS fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:37.834928  4772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f228c374a7634467987eaac91c718aac"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
mode: GRACEFUL
new_leader_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:40032
I20260709 13:14:37.835345  4772 raft_consensus.cc:606] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Received request to transfer leadership to TS 86c683428f324c35a94717ce212f7276
I20260709 13:14:37.836346 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:37.836833 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:37.837371 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:37.910156  5053 raft_consensus.cc:993] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6: : Instructing follower 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:37.910955  5053 raft_consensus.cc:1081] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Signalling peer 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:37.912283  4698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f228c374a7634467987eaac91c718aac"
dest_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:50164
I20260709 13:14:37.912693  4698 raft_consensus.cc:493] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.912911  4698 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.917263  4698 raft_consensus.cc:515] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:37.918537  4698 leader_election.cc:290] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:37.919387  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:37.919755  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:37.920071  4978 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.920395  4772 raft_consensus.cc:3055] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:37.920706  4772 raft_consensus.cc:740] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:37.921340  4772 consensus_queue.cc:260] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:37.922165  4772 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.925201  4978 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:37.926163  4663 leader_election.cc:304] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:37.926638  4772 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:37.926800  4848 raft_consensus.cc:2804] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:37.927304  4848 raft_consensus.cc:697] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:37.928066  4848 consensus_queue.cc:237] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:37.934762  4519 catalog_manager.cc:5697] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 2 to 3, leader changed from fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67) to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 3 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:38.116304  5030 raft_consensus.cc:993] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276: : Instructing follower fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:38.116786  4848 raft_consensus.cc:1081] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Signalling peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:38.118036  4772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e59d5484fea940eab3ee94cc6dadfea5"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:48414
I20260709 13:14:38.118461  4772 raft_consensus.cc:493] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:38.118726  4772 raft_consensus.cc:3060] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:38.124222  4772 raft_consensus.cc:515] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:38.125591  4772 leader_election.cc:290] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:38.126408  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59d5484fea940eab3ee94cc6dadfea5" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:38.126821  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59d5484fea940eab3ee94cc6dadfea5" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:38.127081  4978 raft_consensus.cc:3060] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:38.127460  4698 raft_consensus.cc:3055] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:38.127735  4698 raft_consensus.cc:740] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:38.128487  4698 consensus_queue.cc:260] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:38.129658  4698 raft_consensus.cc:3060] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:38.132725  4978 raft_consensus.cc:2468] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:38.133703  4737 leader_election.cc:304] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:38.134429  5053 raft_consensus.cc:2804] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:38.134588  4698 raft_consensus.cc:2468] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:38.134931  5053 raft_consensus.cc:697] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:38.135883  5053 consensus_queue.cc:237] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:38.142603  4519 catalog_manager.cc:5697] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 1 to 2, leader changed from 86c683428f324c35a94717ce212f7276 (127.27.63.66) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:38.149633  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:14:38.207998  5030 raft_consensus.cc:993] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276: : Instructing follower fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:38.208508  4848 raft_consensus.cc:1081] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Signalling peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:38.209760  4772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:48414
I20260709 13:14:38.210217  4772 raft_consensus.cc:493] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:38.210490  4772 raft_consensus.cc:3060] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:38.214833  4772 raft_consensus.cc:515] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:38.216246  4772 leader_election.cc:290] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:38.217080  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:38.217208  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b07adfc14da14b3faced311b3a96e2a4" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:38.217772  4978 raft_consensus.cc:3060] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:38.217799  4698 raft_consensus.cc:3055] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:38.218298  4698 raft_consensus.cc:740] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:38.219098  4698 consensus_queue.cc:260] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:38.220300  4698 raft_consensus.cc:3060] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:38.224610  4978 raft_consensus.cc:2468] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:38.225764  4737 leader_election.cc:304] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:38.226470  4698 raft_consensus.cc:2468] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:38.226490  5053 raft_consensus.cc:2804] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:38.227324  5053 raft_consensus.cc:697] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:38.228363  5053 consensus_queue.cc:237] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:38.234970  4519 catalog_manager.cc:5697] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 1 to 2, leader changed from 86c683428f324c35a94717ce212f7276 (127.27.63.66) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:38.323752  4772 raft_consensus.cc:1275] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:38.324820  4848 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:38.333128  4978 raft_consensus.cc:1275] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:38.334621  4848 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:38.683465  4978 raft_consensus.cc:1275] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:38.684661  5053 consensus_queue.cc:1048] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.693003  4698 raft_consensus.cc:1275] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:38.694402  5054 consensus_queue.cc:1048] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.743782  4978 raft_consensus.cc:1275] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:38.744942  5054 consensus_queue.cc:1048] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.756333  4698 raft_consensus.cc:1275] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:38.757706  5068 consensus_queue.cc:1048] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.843441 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:38.850543  4772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
mode: GRACEFUL
new_leader_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:40032
I20260709 13:14:38.851095  4772 raft_consensus.cc:606] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Received request to transfer leadership to TS 86c683428f324c35a94717ce212f7276
I20260709 13:14:38.853324  4772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
mode: GRACEFUL
new_leader_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:40032
I20260709 13:14:38.853773  4772 raft_consensus.cc:606] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Received request to transfer leadership to TS 86c683428f324c35a94717ce212f7276
I20260709 13:14:38.855697  4698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f228c374a7634467987eaac91c718aac"
dest_uuid: "86c683428f324c35a94717ce212f7276"
mode: GRACEFUL
new_leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:14:38.856150  4698 raft_consensus.cc:606] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Received request to transfer leadership to TS fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:38.857054 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:38.857533 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:38.858086 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:39.019804  5054 raft_consensus.cc:993] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: : Instructing follower 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:39.020375  5068 raft_consensus.cc:1081] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Signalling peer 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:39.021711  4698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44"
dest_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:50164
I20260709 13:14:39.022148  4698 raft_consensus.cc:493] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:39.022413  4698 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:39.027575  4698 raft_consensus.cc:515] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.029160  4698 leader_election.cc:290] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:39.030072  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:39.029974  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:39.030721  4772 raft_consensus.cc:3055] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:39.030805  4978 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:39.031087  4772 raft_consensus.cc:740] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:39.031848  4772 consensus_queue.cc:260] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.033020  4772 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:39.037724  4978 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:39.038973  4663 leader_election.cc:304] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:39.039417  4772 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:39.039755  5030 raft_consensus.cc:2804] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:39.040283  5030 raft_consensus.cc:697] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:39.041064  5030 consensus_queue.cc:237] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.047015  4519 catalog_manager.cc:5697] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 2 to 3, leader changed from fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67) to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 3 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:39.089982  5054 raft_consensus.cc:993] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: : Instructing follower 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:39.090428  5068 raft_consensus.cc:1081] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Signalling peer 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:39.091805  4698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9"
dest_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:50164
I20260709 13:14:39.092240  4698 raft_consensus.cc:493] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:39.092532  4698 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:39.096688  4698 raft_consensus.cc:515] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.098024  4698 leader_election.cc:290] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:39.098749  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:39.098904  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e3af0616a047368a0ca6f6d13a72e9" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:39.099352  4978 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:39.099480  4772 raft_consensus.cc:3055] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:39.099900  4772 raft_consensus.cc:740] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:39.100616  4772 consensus_queue.cc:260] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.101740  4772 raft_consensus.cc:3060] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:39.106307  4978 raft_consensus.cc:2468] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:39.106789  4772 raft_consensus.cc:2468] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 3.
I20260709 13:14:39.107323  4663 leader_election.cc:304] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:39.108111  5030 raft_consensus.cc:2804] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:39.108572  5030 raft_consensus.cc:697] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:39.109251  5030 consensus_queue.cc:237] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.115772  4519 catalog_manager.cc:5697] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 2 to 3, leader changed from fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67) to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 3 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:39.150610  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:14:39.186909  5052 raft_consensus.cc:993] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276: : Instructing follower fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:39.187469  5052 raft_consensus.cc:1081] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Signalling peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:39.189024  4772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f228c374a7634467987eaac91c718aac"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:48414
I20260709 13:14:39.189558  4772 raft_consensus.cc:493] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:39.189845  4772 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:39.194290  4772 raft_consensus.cc:515] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:39.195631  4772 leader_election.cc:290] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 4 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:39.196475  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:39.196681  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:39.197129  4978 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:39.197201  4698 raft_consensus.cc:3055] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:39.197594  4698 raft_consensus.cc:740] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:39.198215  4698 consensus_queue.cc:260] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:39.199057  4698 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:39.202392  4978 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 4.
I20260709 13:14:39.203488  4737 leader_election.cc:304] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:39.204140  4698 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 4.
I20260709 13:14:39.204334  5068 raft_consensus.cc:2804] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:39.204936  5068 raft_consensus.cc:697] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:39.205624  5068 consensus_queue.cc:237] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:39.211728  4519 catalog_manager.cc:5697] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 3 to 4, leader changed from 86c683428f324c35a94717ce212f7276 (127.27.63.66) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 4 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:39.458012  4978 raft_consensus.cc:1275] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:39.459172  5052 consensus_queue.cc:1048] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.469267  4772 raft_consensus.cc:1275] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:39.470696  5080 consensus_queue.cc:1048] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.548218  4978 raft_consensus.cc:1275] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:39.549429  5080 consensus_queue.cc:1048] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.557825  4772 raft_consensus.cc:1275] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:39.559223  5080 consensus_queue.cc:1048] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.685897  4698 raft_consensus.cc:1275] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 4 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:39.687013  5068 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:39.694866  4978 raft_consensus.cc:1275] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 4 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:39.698016  5053 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:39.864200 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:39.871178  4698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a813d5c1b5e5457dbf4abaa598e36f37"
dest_uuid: "86c683428f324c35a94717ce212f7276"
mode: GRACEFUL
new_leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:14:39.871727  4698 raft_consensus.cc:606] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Received request to transfer leadership to TS fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:39.873961  4698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44"
dest_uuid: "86c683428f324c35a94717ce212f7276"
mode: GRACEFUL
new_leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:52808
I20260709 13:14:39.874382  4698 raft_consensus.cc:606] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Received request to transfer leadership to TS fc0459c7fbe14aaa9f384f3a6ca4a9d6
I20260709 13:14:39.876597  4772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f228c374a7634467987eaac91c718aac"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
mode: GRACEFUL
new_leader_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:40032
I20260709 13:14:39.877039  4772 raft_consensus.cc:606] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 LEADER]: Received request to transfer leadership to TS 86c683428f324c35a94717ce212f7276
I20260709 13:14:39.878012 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:39.878459 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:39.878979 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:39.892952  5052 raft_consensus.cc:993] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276: : Instructing follower fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:39.893496  5080 raft_consensus.cc:1081] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Signalling peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:39.894773  4772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a813d5c1b5e5457dbf4abaa598e36f37"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:48414
I20260709 13:14:39.895213  4772 raft_consensus.cc:493] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:39.895496  4772 raft_consensus.cc:3060] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:39.899950  4772 raft_consensus.cc:515] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.902060  4772 leader_election.cc:290] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:39.902115  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a813d5c1b5e5457dbf4abaa598e36f37" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:39.902698  4978 raft_consensus.cc:3060] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:39.902925  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a813d5c1b5e5457dbf4abaa598e36f37" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:39.903520  4698 raft_consensus.cc:3055] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:39.903761  4698 raft_consensus.cc:740] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:39.904456  4698 consensus_queue.cc:260] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.905380  4698 raft_consensus.cc:3060] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:39.907425  4978 raft_consensus.cc:2468] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:39.908362  4737 leader_election.cc:304] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:39.909096  5087 raft_consensus.cc:2804] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:39.909479  5087 raft_consensus.cc:697] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:39.910192  4698 raft_consensus.cc:2468] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 2.
I20260709 13:14:39.910342  5087 consensus_queue.cc:237] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.916674  4519 catalog_manager.cc:5697] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 1 to 2, leader changed from 86c683428f324c35a94717ce212f7276 (127.27.63.66) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:39.959661  5030 raft_consensus.cc:993] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276: : Instructing follower fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:39.960276  5080 raft_consensus.cc:1081] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Signalling peer fc0459c7fbe14aaa9f384f3a6ca4a9d6 to start an election
I20260709 13:14:39.961625  4772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44"
dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
 from {username='slave'} at 127.0.0.1:48414
I20260709 13:14:39.962147  4772 raft_consensus.cc:493] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:39.962466  4772 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:39.967154  4772 raft_consensus.cc:515] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.968495  4772 leader_election.cc:290] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 4 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855)
I20260709 13:14:39.969316  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:39.969480  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd465e5c5472416a917ba1a4bc550a44" candidate_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "86c683428f324c35a94717ce212f7276"
I20260709 13:14:39.969914  4978 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:39.970016  4698 raft_consensus.cc:3055] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:39.970369  4698 raft_consensus.cc:740] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:39.970920  4698 consensus_queue.cc:260] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.971721  4698 raft_consensus.cc:3060] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:39.974438  4978 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 4.
I20260709 13:14:39.975379  4737 leader_election.cc:304] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87819469991457784c7cf663512487e, fc0459c7fbe14aaa9f384f3a6ca4a9d6; no voters: 
I20260709 13:14:39.976119  5087 raft_consensus.cc:2804] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:39.976212  4698 raft_consensus.cc:2468] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0459c7fbe14aaa9f384f3a6ca4a9d6 in term 4.
I20260709 13:14:39.976630  5087 raft_consensus.cc:697] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 LEADER]: Becoming Leader. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:39.977459  5087 consensus_queue.cc:237] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } }
I20260709 13:14:39.982977  4519 catalog_manager.cc:5697] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 reported cstate change: term changed from 3 to 4, leader changed from 86c683428f324c35a94717ce212f7276 (127.27.63.66) to fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67). New cstate: current_term: 4 leader_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:40.151578  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:14:40.193111  5068 raft_consensus.cc:993] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6: : Instructing follower 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:40.193768  5087 raft_consensus.cc:1081] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 LEADER]: Signalling peer 86c683428f324c35a94717ce212f7276 to start an election
I20260709 13:14:40.195182  4698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f228c374a7634467987eaac91c718aac"
dest_uuid: "86c683428f324c35a94717ce212f7276"
 from {username='slave'} at 127.0.0.1:50164
I20260709 13:14:40.195632  4698 raft_consensus.cc:493] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:40.195953  4698 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:40.200273  4698 raft_consensus.cc:515] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:40.201653  4698 leader_election.cc:290] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 5 election: Requested vote from peers c87819469991457784c7cf663512487e (127.27.63.65:35357), fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67:46565)
I20260709 13:14:40.202296  4978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c87819469991457784c7cf663512487e"
I20260709 13:14:40.202935  4978 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:40.202605  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228c374a7634467987eaac91c718aac" candidate_uuid: "86c683428f324c35a94717ce212f7276" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6"
I20260709 13:14:40.203601  4772 raft_consensus.cc:3055] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:40.203878  4772 raft_consensus.cc:740] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fc0459c7fbe14aaa9f384f3a6ca4a9d6, State: Running, Role: LEADER
I20260709 13:14:40.204674  4772 consensus_queue.cc:260] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:40.206950  4772 raft_consensus.cc:3060] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:40.209446  4978 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 5.
I20260709 13:14:40.210580  4663 leader_election.cc:304] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c683428f324c35a94717ce212f7276, c87819469991457784c7cf663512487e; no voters: 
I20260709 13:14:40.211277  5080 raft_consensus.cc:2804] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:40.211700  5080 raft_consensus.cc:697] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 5 LEADER]: Becoming Leader. State: Replica: 86c683428f324c35a94717ce212f7276, State: Running, Role: LEADER
I20260709 13:14:40.212597  4772 raft_consensus.cc:2468] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c683428f324c35a94717ce212f7276 in term 5.
I20260709 13:14:40.212813  5080 consensus_queue.cc:237] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } }
I20260709 13:14:40.219112  4519 catalog_manager.cc:5697] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 reported cstate change: term changed from 4 to 5, leader changed from fc0459c7fbe14aaa9f384f3a6ca4a9d6 (127.27.63.67) to 86c683428f324c35a94717ce212f7276 (127.27.63.66). New cstate: current_term: 5 leader_uuid: "86c683428f324c35a94717ce212f7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.379454  4698 raft_consensus.cc:1275] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:40.380534  5087 consensus_queue.cc:1048] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:40.388813  4699 raft_consensus.cc:1275] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 4 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:40.390256  5053 consensus_queue.cc:1048] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c683428f324c35a94717ce212f7276" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 38855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:40.390906  4978 raft_consensus.cc:1275] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:40.392333  5053 consensus_queue.cc:1048] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:40.404448  4979 raft_consensus.cc:1275] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 4 FOLLOWER]: Refusing update from remote peer fc0459c7fbe14aaa9f384f3a6ca4a9d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:40.411543  5087 consensus_queue.cc:1048] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:40.736624  4772 raft_consensus.cc:1275] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 5 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:14:40.737764  5080 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0459c7fbe14aaa9f384f3a6ca4a9d6" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:40.748140  4979 raft_consensus.cc:1275] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 5 FOLLOWER]: Refusing update from remote peer 86c683428f324c35a94717ce212f7276: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:14:40.749495  5080 consensus_queue.cc:1048] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87819469991457784c7cf663512487e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 35357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:40.884773 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:40.913879 27901 tablet_server.cc:179] TabletServer@127.27.63.65:35357 shutting down...
I20260709 13:14:40.933676 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:40.934269 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.934795 27901 raft_consensus.cc:2243] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.935181 27901 raft_consensus.cc:2272] T a813d5c1b5e5457dbf4abaa598e36f37 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.937307 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.937790 27901 raft_consensus.cc:2243] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.938114 27901 raft_consensus.cc:2272] T 82c2e3df5f3c45bf886fff877b2c8ca4 P c87819469991457784c7cf663512487e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.939657 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.940189 27901 raft_consensus.cc:2243] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.940526 27901 raft_consensus.cc:2272] T 2529f93211584032b0ca59e2beddd84a P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.942245 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.942935 27901 raft_consensus.cc:2243] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.943296 27901 raft_consensus.cc:2272] T 20fcae796dda4168807069fba8c1974a P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.945153 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.945753 27901 raft_consensus.cc:2243] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.946179 27901 raft_consensus.cc:2272] T e59d5484fea940eab3ee94cc6dadfea5 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:40.946241  4737 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
I20260709 13:14:40.948587 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.949229 27901 raft_consensus.cc:2243] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:40.949257  4737 consensus_peers.cc:597] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:40.949841 27901 raft_consensus.cc:2272] T dd465e5c5472416a917ba1a4bc550a44 P c87819469991457784c7cf663512487e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.952432 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.952932 27901 raft_consensus.cc:2243] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.953330 27901 raft_consensus.cc:2272] T f228c374a7634467987eaac91c718aac P c87819469991457784c7cf663512487e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.955314 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.955819 27901 raft_consensus.cc:2243] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.956233 27901 raft_consensus.cc:2272] T 5d09b12ff1334907b9032e5b613a642e P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.957841 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.958290 27901 raft_consensus.cc:2243] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.958657 27901 raft_consensus.cc:2272] T b07adfc14da14b3faced311b3a96e2a4 P c87819469991457784c7cf663512487e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.960302 27901 tablet_replica.cc:333] stopping tablet replica
I20260709 13:14:40.960736 27901 raft_consensus.cc:2243] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.961107 27901 raft_consensus.cc:2272] T 11e3af0616a047368a0ca6f6d13a72e9 P c87819469991457784c7cf663512487e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.988575 27901 tablet_server.cc:196] TabletServer@127.27.63.65:35357 shutdown complete.
I20260709 13:14:41.007129 27901 tablet_server.cc:179] TabletServer@127.27.63.66:0 shutting down...
W20260709 13:14:41.014891  4736 consensus_peers.cc:597] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855): Couldn't send request to peer 86c683428f324c35a94717ce212f7276. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.030287 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:41.031035 27901 tablet_replica.cc:333] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.031720 27901 raft_consensus.cc:2243] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.032176 27901 raft_consensus.cc:2272] T b07adfc14da14b3faced311b3a96e2a4 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.034168 27901 tablet_replica.cc:333] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.034724 27901 raft_consensus.cc:2243] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.035084 27901 raft_consensus.cc:2272] T dd465e5c5472416a917ba1a4bc550a44 P 86c683428f324c35a94717ce212f7276 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.037382 27901 tablet_replica.cc:333] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.037917 27901 raft_consensus.cc:2243] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.038269 27901 raft_consensus.cc:2272] T e59d5484fea940eab3ee94cc6dadfea5 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.040689 27901 tablet_replica.cc:333] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.041244 27901 raft_consensus.cc:2243] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.041958 27901 raft_consensus.cc:2272] T 11e3af0616a047368a0ca6f6d13a72e9 P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.043955 27901 tablet_replica.cc:333] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.044510 27901 raft_consensus.cc:2243] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.045235 27901 raft_consensus.cc:2272] T f228c374a7634467987eaac91c718aac P 86c683428f324c35a94717ce212f7276 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.047777 27901 tablet_replica.cc:333] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.048413 27901 raft_consensus.cc:2243] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.049139 27901 raft_consensus.cc:2272] T 5d09b12ff1334907b9032e5b613a642e P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.051075 27901 tablet_replica.cc:333] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.051601 27901 raft_consensus.cc:2243] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.052949 27901 raft_consensus.cc:2272] T 82c2e3df5f3c45bf886fff877b2c8ca4 P 86c683428f324c35a94717ce212f7276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.054863 27901 tablet_replica.cc:333] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.055938 27901 raft_consensus.cc:2243] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.056349 27901 raft_consensus.cc:2272] T a813d5c1b5e5457dbf4abaa598e36f37 P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.058709 27901 tablet_replica.cc:333] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.059196 27901 raft_consensus.cc:2243] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.059904 27901 raft_consensus.cc:2272] T 2529f93211584032b0ca59e2beddd84a P 86c683428f324c35a94717ce212f7276 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.061846 27901 tablet_replica.cc:333] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276: stopping tablet replica
I20260709 13:14:41.062304 27901 raft_consensus.cc:2243] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.062738 27901 raft_consensus.cc:2272] T 20fcae796dda4168807069fba8c1974a P 86c683428f324c35a94717ce212f7276 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:41.088900  4736 consensus_peers.cc:597] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855): Couldn't send request to peer 86c683428f324c35a94717ce212f7276. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:38855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.092897 27901 tablet_server.cc:196] TabletServer@127.27.63.66:0 shutdown complete.
W20260709 13:14:41.100164  4736 consensus_peers.cc:597] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer 86c683428f324c35a94717ce212f7276 (127.27.63.66:38855): Couldn't send request to peer 86c683428f324c35a94717ce212f7276. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:38855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.117105 27901 tablet_server.cc:179] TabletServer@127.27.63.67:0 shutting down...
W20260709 13:14:41.137992  4737 consensus_peers.cc:597] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:41.144124  4737 consensus_peers.cc:597] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 -> Peer c87819469991457784c7cf663512487e (127.27.63.65:35357): Couldn't send request to peer c87819469991457784c7cf663512487e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:35357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.148705 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:41.149367 27901 tablet_replica.cc:333] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.149971 27901 raft_consensus.cc:2243] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.150410 27901 raft_consensus.cc:2272] T 82c2e3df5f3c45bf886fff877b2c8ca4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.152467 27901 tablet_replica.cc:333] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
W20260709 13:14:41.152527  4570 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:14:41.153146 27901 raft_consensus.cc:2243] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.154007 27901 raft_consensus.cc:2272] T dd465e5c5472416a917ba1a4bc550a44 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.156497 27901 tablet_replica.cc:333] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.156996 27901 raft_consensus.cc:2243] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.157691 27901 raft_consensus.cc:2272] T e59d5484fea940eab3ee94cc6dadfea5 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.159622 27901 tablet_replica.cc:333] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.160190 27901 raft_consensus.cc:2243] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.160581 27901 raft_consensus.cc:2272] T 11e3af0616a047368a0ca6f6d13a72e9 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.162365 27901 tablet_replica.cc:333] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.162868 27901 raft_consensus.cc:2243] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.163246 27901 raft_consensus.cc:2272] T 5d09b12ff1334907b9032e5b613a642e P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.165180 27901 tablet_replica.cc:333] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.165666 27901 raft_consensus.cc:2243] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.166043 27901 raft_consensus.cc:2272] T f228c374a7634467987eaac91c718aac P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.168385 27901 tablet_replica.cc:333] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.168867 27901 raft_consensus.cc:2243] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.169605 27901 raft_consensus.cc:2272] T 20fcae796dda4168807069fba8c1974a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.171558 27901 tablet_replica.cc:333] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.172032 27901 raft_consensus.cc:2243] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.172462 27901 raft_consensus.cc:2272] T 2529f93211584032b0ca59e2beddd84a P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.174242 27901 tablet_replica.cc:333] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.174716 27901 raft_consensus.cc:2243] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.175383 27901 raft_consensus.cc:2272] T a813d5c1b5e5457dbf4abaa598e36f37 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.177526 27901 tablet_replica.cc:333] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6: stopping tablet replica
I20260709 13:14:41.177994 27901 raft_consensus.cc:2243] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.178691 27901 raft_consensus.cc:2272] T b07adfc14da14b3faced311b3a96e2a4 P fc0459c7fbe14aaa9f384f3a6ca4a9d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.208168 27901 tablet_server.cc:196] TabletServer@127.27.63.67:0 shutdown complete.
I20260709 13:14:41.230540 27901 master.cc:562] Master@127.27.63.126:39157 shutting down...
I20260709 13:14:41.255209 27901 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.255769 27901 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.256227 27901 tablet_replica.cc:333] T 00000000000000000000000000000000 P e18c1581fb074325a75c30549205fd8e: stopping tablet replica
I20260709 13:14:41.277614 27901 master.cc:584] Master@127.27.63.126:39157 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (14341 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:14:41.321491 27901 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.63.126:46141
I20260709 13:14:41.322537 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:41.327828  5110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:41.328612  5113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:41.327889  5111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:41.329742 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:41.330595 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:41.330792 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:41.330937 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602881330920 us; error 0 us; skew 500 ppm
I20260709 13:14:41.331446 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:41.333707 27901 webserver.cc:533] Webserver started at http://127.27.63.126:44319/ using document root <none> and password file <none>
I20260709 13:14:41.334182 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:41.334378 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:41.334648 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:41.335702 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/master-0-root/instance:
uuid: "37c6f9655e464423b1635630bd3a09d2"
format_stamp: "Formatted at 2026-07-09 13:14:41 on dist-test-slave-sjgx"
I20260709 13:14:41.340233 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:14:41.343431  5118 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.344173 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:14:41.344444 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/master-0-root
uuid: "37c6f9655e464423b1635630bd3a09d2"
format_stamp: "Formatted at 2026-07-09 13:14:41 on dist-test-slave-sjgx"
I20260709 13:14:41.344738 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:41.358007 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:41.359172 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:41.395835 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.126:46141
I20260709 13:14:41.395936  5169 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.126:46141 every 8 connection(s)
I20260709 13:14:41.401597  5170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:41.414163  5170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2: Bootstrap starting.
I20260709 13:14:41.418911  5170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:41.422942  5170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2: No bootstrap required, opened a new log
I20260709 13:14:41.425061  5170 raft_consensus.cc:359] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c6f9655e464423b1635630bd3a09d2" member_type: VOTER }
I20260709 13:14:41.425457  5170 raft_consensus.cc:385] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:41.425683  5170 raft_consensus.cc:740] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c6f9655e464423b1635630bd3a09d2, State: Initialized, Role: FOLLOWER
I20260709 13:14:41.426319  5170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c6f9655e464423b1635630bd3a09d2" member_type: VOTER }
I20260709 13:14:41.426752  5170 raft_consensus.cc:399] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:41.426980  5170 raft_consensus.cc:493] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:41.427237  5170 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:41.431770  5170 raft_consensus.cc:515] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c6f9655e464423b1635630bd3a09d2" member_type: VOTER }
I20260709 13:14:41.432273  5170 leader_election.cc:304] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37c6f9655e464423b1635630bd3a09d2; no voters: 
I20260709 13:14:41.433319  5170 leader_election.cc:290] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:41.433655  5173 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:41.434958  5173 raft_consensus.cc:697] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 1 LEADER]: Becoming Leader. State: Replica: 37c6f9655e464423b1635630bd3a09d2, State: Running, Role: LEADER
I20260709 13:14:41.435587  5173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c6f9655e464423b1635630bd3a09d2" member_type: VOTER }
I20260709 13:14:41.436141  5170 sys_catalog.cc:565] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:41.440357  5175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37c6f9655e464423b1635630bd3a09d2. Latest consensus state: current_term: 1 leader_uuid: "37c6f9655e464423b1635630bd3a09d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c6f9655e464423b1635630bd3a09d2" member_type: VOTER } }
I20260709 13:14:41.441044  5175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:41.439888  5174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37c6f9655e464423b1635630bd3a09d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c6f9655e464423b1635630bd3a09d2" member_type: VOTER } }
I20260709 13:14:41.442104  5174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:41.443060  5180 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:41.451458  5180 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:41.455545 27901 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:41.462246  5180 catalog_manager.cc:1383] Generated new cluster ID: 061c26275d33412da0d851355e331094
I20260709 13:14:41.462587  5180 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:41.482323  5180 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:41.483625  5180 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:41.499528  5180 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2: Generated new TSK 0
I20260709 13:14:41.500208  5180 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:41.525270 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:41.531687  5191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:41.533049  5192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:41.534798  5194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:41.535728 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:41.536657 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:41.536844 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:41.537072 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602881537026 us; error 0 us; skew 500 ppm
I20260709 13:14:41.537669 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:41.539955 27901 webserver.cc:533] Webserver started at http://127.27.63.65:46287/ using document root <none> and password file <none>
I20260709 13:14:41.540530 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:41.540728 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:41.541009 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:41.542070 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-0-root/instance:
uuid: "886a95f2f01b4530a41ae230bac8cf47"
format_stamp: "Formatted at 2026-07-09 13:14:41 on dist-test-slave-sjgx"
I20260709 13:14:41.546828 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:14:41.550529  5199 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.551309 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:14:41.551725 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "886a95f2f01b4530a41ae230bac8cf47"
format_stamp: "Formatted at 2026-07-09 13:14:41 on dist-test-slave-sjgx"
I20260709 13:14:41.552095 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:41.568893 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:41.570252 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:41.571849 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:41.574324 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:41.574573 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.574834 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:41.575034 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.618860 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:36947
I20260709 13:14:41.619004  5261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:36947 every 8 connection(s)
I20260709 13:14:41.624425 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:41.631419  5266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:41.632582  5267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:41.634733  5269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:41.635782 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:41.637212 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:41.637490 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:41.637694 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602881637648 us; error 0 us; skew 500 ppm
I20260709 13:14:41.638240  5262 heartbeater.cc:344] Connected to a master server at 127.27.63.126:46141
I20260709 13:14:41.638448 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:41.638695  5262 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:41.639905  5262 heartbeater.cc:507] Master 127.27.63.126:46141 requested a full tablet report, sending...
I20260709 13:14:41.641695 27901 webserver.cc:533] Webserver started at http://127.27.63.66:35251/ using document root <none> and password file <none>
I20260709 13:14:41.642405 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:41.642616 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:41.642663  5135 ts_manager.cc:194] Registered new tserver with Master: 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947)
I20260709 13:14:41.642952 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:41.644263 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-1-root/instance:
uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
format_stamp: "Formatted at 2026-07-09 13:14:41 on dist-test-slave-sjgx"
I20260709 13:14:41.644902  5135 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38232
I20260709 13:14:41.649350 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:14:41.653100  5274 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.654023 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:14:41.654335 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-1-root
uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
format_stamp: "Formatted at 2026-07-09 13:14:41 on dist-test-slave-sjgx"
I20260709 13:14:41.654613 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:41.676950 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:41.678161 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:41.679677 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:41.681881 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:41.682051 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.682309 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:41.682488 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.726367 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.66:33827
I20260709 13:14:41.726536  5336 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.66:33827 every 8 connection(s)
I20260709 13:14:41.731233 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:41.738103  5340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:41.739784  5341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:41.742129 27901 server_base.cc:1061] running on GCE node
W20260709 13:14:41.742628  5343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:41.743340  5337 heartbeater.cc:344] Connected to a master server at 127.27.63.126:46141
I20260709 13:14:41.743794  5337 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:41.743853 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:41.744228 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:41.744448 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602881744429 us; error 0 us; skew 500 ppm
I20260709 13:14:41.744771  5337 heartbeater.cc:507] Master 127.27.63.126:46141 requested a full tablet report, sending...
I20260709 13:14:41.745266 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:41.747422  5135 ts_manager.cc:194] Registered new tserver with Master: 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827)
I20260709 13:14:41.748901 27901 webserver.cc:533] Webserver started at http://127.27.63.67:37865/ using document root <none> and password file <none>
I20260709 13:14:41.749187  5135 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38244
I20260709 13:14:41.749536 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:41.749891 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:41.750190 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:41.751763 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-2-root/instance:
uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
format_stamp: "Formatted at 2026-07-09 13:14:41 on dist-test-slave-sjgx"
I20260709 13:14:41.756453 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:14:41.760571  5348 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.761404 27901 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:14:41.761868 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-2-root
uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
format_stamp: "Formatted at 2026-07-09 13:14:41 on dist-test-slave-sjgx"
I20260709 13:14:41.762151 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602591602838-27901-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:41.776036 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:41.777477 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:41.779047 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:41.781383 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:41.781635 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.781898 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:41.782073 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:41.829828 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.67:39263
I20260709 13:14:41.830001  5410 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.67:39263 every 8 connection(s)
I20260709 13:14:41.845894  5411 heartbeater.cc:344] Connected to a master server at 127.27.63.126:46141
I20260709 13:14:41.846372  5411 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:41.847222  5411 heartbeater.cc:507] Master 127.27.63.126:46141 requested a full tablet report, sending...
I20260709 13:14:41.849277  5135 ts_manager.cc:194] Registered new tserver with Master: 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:41.849581 27901 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016052619s
I20260709 13:14:41.850790  5135 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38246
I20260709 13:14:41.922657  5135 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38254:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:41.925880  5135 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:41.997112  5227 tablet_service.cc:1511] Processing CreateTablet for tablet 2cea52ce538b4f08ad59722a3658a9b8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=f84b15a0806e43b198606cea2316640b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:14:41.999264  5227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cea52ce538b4f08ad59722a3658a9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:41.999554  5302 tablet_service.cc:1511] Processing CreateTablet for tablet 2cea52ce538b4f08ad59722a3658a9b8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=f84b15a0806e43b198606cea2316640b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:14:42.000352  5301 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3bb62c7afc4316ad9100a43f9e5ef2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=f84b15a0806e43b198606cea2316640b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:14:42.001147  5302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cea52ce538b4f08ad59722a3658a9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:41.997316  5226 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3bb62c7afc4316ad9100a43f9e5ef2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=f84b15a0806e43b198606cea2316640b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:14:42.002002  5301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3bb62c7afc4316ad9100a43f9e5ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.002489  5226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3bb62c7afc4316ad9100a43f9e5ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.009099  5376 tablet_service.cc:1511] Processing CreateTablet for tablet 2cea52ce538b4f08ad59722a3658a9b8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=f84b15a0806e43b198606cea2316640b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:14:42.011056  5376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cea52ce538b4f08ad59722a3658a9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.014063  5375 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3bb62c7afc4316ad9100a43f9e5ef2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=f84b15a0806e43b198606cea2316640b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:14:42.015542  5375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3bb62c7afc4316ad9100a43f9e5ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.041365  5427 tablet_bootstrap.cc:492] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Bootstrap starting.
I20260709 13:14:42.048341  5427 tablet_bootstrap.cc:654] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.058615  5428 tablet_bootstrap.cc:492] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47: Bootstrap starting.
I20260709 13:14:42.061074  5427 tablet_bootstrap.cc:492] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895: No bootstrap required, opened a new log
I20260709 13:14:42.061674  5427 ts_tablet_manager.cc:1403] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:14:42.062803  5430 tablet_bootstrap.cc:492] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Bootstrap starting.
I20260709 13:14:42.069912  5428 tablet_bootstrap.cc:654] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.070333  5430 tablet_bootstrap.cc:654] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.073773  5427 raft_consensus.cc:359] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.074972  5427 raft_consensus.cc:385] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.075287  5427 raft_consensus.cc:740] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.075556  5430 tablet_bootstrap.cc:492] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21: No bootstrap required, opened a new log
I20260709 13:14:42.075663  5428 tablet_bootstrap.cc:492] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47: No bootstrap required, opened a new log
I20260709 13:14:42.076303  5430 ts_tablet_manager.cc:1403] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:14:42.076350  5428 ts_tablet_manager.cc:1403] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 13:14:42.076205  5427 consensus_queue.cc:260] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.079263  5337 heartbeater.cc:499] Master 127.27.63.126:46141 was elected leader, sending a full tablet report...
I20260709 13:14:42.079656  5427 ts_tablet_manager.cc:1434] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:14:42.079665  5430 raft_consensus.cc:359] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.080139  5428 raft_consensus.cc:359] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.080739  5430 raft_consensus.cc:385] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.080854  5427 tablet_bootstrap.cc:492] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Bootstrap starting.
I20260709 13:14:42.080947  5428 raft_consensus.cc:385] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.081123  5430 raft_consensus.cc:740] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd40bda09114c6e92fe9b0e0b0cfe21, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.081522  5428 raft_consensus.cc:740] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.082172  5430 consensus_queue.cc:260] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.082412  5428 consensus_queue.cc:260] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.089983  5427 tablet_bootstrap.cc:654] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.104084  5427 tablet_bootstrap.cc:492] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895: No bootstrap required, opened a new log
I20260709 13:14:42.104112  5411 heartbeater.cc:499] Master 127.27.63.126:46141 was elected leader, sending a full tablet report...
I20260709 13:14:42.104777  5430 ts_tablet_manager.cc:1434] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Time spent starting tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 13:14:42.104928  5427 ts_tablet_manager.cc:1403] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.008s
I20260709 13:14:42.105834  5430 tablet_bootstrap.cc:492] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Bootstrap starting.
I20260709 13:14:42.109326  5427 raft_consensus.cc:359] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.110342  5427 raft_consensus.cc:385] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.110716  5427 raft_consensus.cc:740] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.111673  5427 consensus_queue.cc:260] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.113636  5430 tablet_bootstrap.cc:654] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.114574  5427 ts_tablet_manager.cc:1434] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:14:42.118187  5262 heartbeater.cc:499] Master 127.27.63.126:46141 was elected leader, sending a full tablet report...
I20260709 13:14:42.118570  5428 ts_tablet_manager.cc:1434] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47: Time spent starting tablet: real 0.042s	user 0.026s	sys 0.012s
I20260709 13:14:42.119575  5428 tablet_bootstrap.cc:492] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47: Bootstrap starting.
I20260709 13:14:42.126729  5428 tablet_bootstrap.cc:654] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.128690  5430 tablet_bootstrap.cc:492] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21: No bootstrap required, opened a new log
I20260709 13:14:42.129282  5430 ts_tablet_manager.cc:1403] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.008s
I20260709 13:14:42.132339  5430 raft_consensus.cc:359] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.133137  5430 raft_consensus.cc:385] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.133473  5430 raft_consensus.cc:740] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd40bda09114c6e92fe9b0e0b0cfe21, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.134443  5430 consensus_queue.cc:260] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.135471  5428 tablet_bootstrap.cc:492] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47: No bootstrap required, opened a new log
I20260709 13:14:42.136020  5428 ts_tablet_manager.cc:1403] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 13:14:42.136584  5430 ts_tablet_manager.cc:1434] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.006s
I20260709 13:14:42.138906  5428 raft_consensus.cc:359] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.139680  5428 raft_consensus.cc:385] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.139914  5428 raft_consensus.cc:740] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.140702  5428 consensus_queue.cc:260] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.142679  5428 ts_tablet_manager.cc:1434] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:14:42.205580  5435 raft_consensus.cc:493] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.206059  5435 raft_consensus.cc:515] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.208405  5435 leader_election.cc:290] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.218679  5312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3bb62c7afc4316ad9100a43f9e5ef2" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" is_pre_election: true
I20260709 13:14:42.219103  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3bb62c7afc4316ad9100a43f9e5ef2" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" is_pre_election: true
I20260709 13:14:42.219485  5312 raft_consensus.cc:2468] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 0.
I20260709 13:14:42.219856  5386 raft_consensus.cc:2468] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 0.
I20260709 13:14:42.220785  5201 leader_election.cc:304] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 886a95f2f01b4530a41ae230bac8cf47; no voters: 
I20260709 13:14:42.221585  5435 raft_consensus.cc:2804] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:42.221916  5435 raft_consensus.cc:493] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.222138  5435 raft_consensus.cc:3060] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.228070  5435 raft_consensus.cc:515] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.229735  5435 leader_election.cc:290] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 1 election: Requested vote from peers 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.230469  5312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3bb62c7afc4316ad9100a43f9e5ef2" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
I20260709 13:14:42.230772  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3bb62c7afc4316ad9100a43f9e5ef2" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
I20260709 13:14:42.231127  5312 raft_consensus.cc:3060] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.231351  5386 raft_consensus.cc:3060] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.240067  5386 raft_consensus.cc:2468] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 1.
I20260709 13:14:42.240067  5312 raft_consensus.cc:2468] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 1.
I20260709 13:14:42.241387  5203 leader_election.cc:304] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dd40bda09114c6e92fe9b0e0b0cfe21, 886a95f2f01b4530a41ae230bac8cf47; no voters: 
I20260709 13:14:42.242728  5435 raft_consensus.cc:2804] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:42.243582  5435 raft_consensus.cc:697] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 LEADER]: Becoming Leader. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Running, Role: LEADER
I20260709 13:14:42.244547  5435 consensus_queue.cc:237] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.254346  5133 catalog_manager.cc:5697] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "886a95f2f01b4530a41ae230bac8cf47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:42.306797  5433 raft_consensus.cc:493] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.307307  5433 raft_consensus.cc:515] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.309577  5433 leader_election.cc:290] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.321810  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cea52ce538b4f08ad59722a3658a9b8" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" is_pre_election: true
I20260709 13:14:42.321913  5237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cea52ce538b4f08ad59722a3658a9b8" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "886a95f2f01b4530a41ae230bac8cf47" is_pre_election: true
I20260709 13:14:42.322800  5386 raft_consensus.cc:2468] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 0.
I20260709 13:14:42.322856  5237 raft_consensus.cc:2468] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 0.
I20260709 13:14:42.323915  5278 leader_election.cc:304] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 5dd40bda09114c6e92fe9b0e0b0cfe21; no voters: 
I20260709 13:14:42.324739  5433 raft_consensus.cc:2804] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:42.325122  5433 raft_consensus.cc:493] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.325389  5433 raft_consensus.cc:3060] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.331043  5433 raft_consensus.cc:515] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.332679  5433 leader_election.cc:290] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 1 election: Requested vote from peers 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.333509  5237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cea52ce538b4f08ad59722a3658a9b8" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
I20260709 13:14:42.333657  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cea52ce538b4f08ad59722a3658a9b8" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
I20260709 13:14:42.334066  5237 raft_consensus.cc:3060] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.334245  5386 raft_consensus.cc:3060] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.339808  5386 raft_consensus.cc:2468] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 1.
I20260709 13:14:42.339875  5237 raft_consensus.cc:2468] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 1.
I20260709 13:14:42.340915  5278 leader_election.cc:304] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 5dd40bda09114c6e92fe9b0e0b0cfe21; no voters: 
I20260709 13:14:42.341589  5433 raft_consensus.cc:2804] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:42.342690  5433 raft_consensus.cc:697] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Becoming Leader. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Running, Role: LEADER
I20260709 13:14:42.343519  5433 consensus_queue.cc:237] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.350236  5133 catalog_manager.cc:5697] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:42.398103  5133 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38270:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:42.400035  5133 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:42.423435  5226 tablet_service.cc:1511] Processing CreateTablet for tablet 03bc9437fb0e4cb1ab2859bed67be033 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ae090acd7eef42e489bfaf4e244338a2]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:14:42.424722  5226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03bc9437fb0e4cb1ab2859bed67be033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.424762  5302 tablet_service.cc:1511] Processing CreateTablet for tablet 03bc9437fb0e4cb1ab2859bed67be033 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ae090acd7eef42e489bfaf4e244338a2]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:14:42.426064  5302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03bc9437fb0e4cb1ab2859bed67be033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.426088  5375 tablet_service.cc:1511] Processing CreateTablet for tablet 03bc9437fb0e4cb1ab2859bed67be033 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ae090acd7eef42e489bfaf4e244338a2]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:14:42.427397  5375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03bc9437fb0e4cb1ab2859bed67be033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.428771  5227 tablet_service.cc:1511] Processing CreateTablet for tablet dc10ad6008dc497aafb97e8e77bc0581 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ae090acd7eef42e489bfaf4e244338a2]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:14:42.430007  5227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc10ad6008dc497aafb97e8e77bc0581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.433712  5301 tablet_service.cc:1511] Processing CreateTablet for tablet dc10ad6008dc497aafb97e8e77bc0581 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ae090acd7eef42e489bfaf4e244338a2]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:14:42.434983  5301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc10ad6008dc497aafb97e8e77bc0581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.435608  5376 tablet_service.cc:1511] Processing CreateTablet for tablet dc10ad6008dc497aafb97e8e77bc0581 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ae090acd7eef42e489bfaf4e244338a2]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:14:42.436892  5376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc10ad6008dc497aafb97e8e77bc0581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.441628  5428 tablet_bootstrap.cc:492] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47: Bootstrap starting.
I20260709 13:14:42.448499  5428 tablet_bootstrap.cc:654] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:42.450531  5183 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:42.452237  5427 tablet_bootstrap.cc:492] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Bootstrap starting.
I20260709 13:14:42.459714  5427 tablet_bootstrap.cc:654] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.467121  5430 tablet_bootstrap.cc:492] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Bootstrap starting.
I20260709 13:14:42.476883  5428 tablet_bootstrap.cc:492] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47: No bootstrap required, opened a new log
I20260709 13:14:42.477479  5428 ts_tablet_manager.cc:1403] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.013s
I20260709 13:14:42.480510  5428 raft_consensus.cc:359] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.481195  5430 tablet_bootstrap.cc:654] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.481240  5428 raft_consensus.cc:385] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.481813  5428 raft_consensus.cc:740] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.482892  5427 tablet_bootstrap.cc:492] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895: No bootstrap required, opened a new log
I20260709 13:14:42.482863  5428 consensus_queue.cc:260] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.483489  5427 ts_tablet_manager.cc:1403] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.015s
I20260709 13:14:42.485117  5428 ts_tablet_manager.cc:1434] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:14:42.486121  5428 tablet_bootstrap.cc:492] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47: Bootstrap starting.
I20260709 13:14:42.487352  5427 raft_consensus.cc:359] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.488376  5427 raft_consensus.cc:385] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.488754  5427 raft_consensus.cc:740] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.489749  5427 consensus_queue.cc:260] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.492165  5427 ts_tablet_manager.cc:1434] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Time spent starting tablet: real 0.008s	user 0.011s	sys 0.000s
I20260709 13:14:42.492964  5430 tablet_bootstrap.cc:492] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21: No bootstrap required, opened a new log
I20260709 13:14:42.493366  5428 tablet_bootstrap.cc:654] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.493480  5430 ts_tablet_manager.cc:1403] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Time spent bootstrapping tablet: real 0.027s	user 0.021s	sys 0.000s
I20260709 13:14:42.494932  5427 tablet_bootstrap.cc:492] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Bootstrap starting.
I20260709 13:14:42.497215  5430 raft_consensus.cc:359] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.497922  5430 raft_consensus.cc:385] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.498217  5430 raft_consensus.cc:740] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd40bda09114c6e92fe9b0e0b0cfe21, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.499120  5430 consensus_queue.cc:260] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.503527  5427 tablet_bootstrap.cc:654] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.507282  5430 ts_tablet_manager.cc:1434] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:14:42.508769  5430 tablet_bootstrap.cc:492] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Bootstrap starting.
I20260709 13:14:42.515141  5427 tablet_bootstrap.cc:492] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895: No bootstrap required, opened a new log
I20260709 13:14:42.515691  5427 ts_tablet_manager.cc:1403] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.008s
I20260709 13:14:42.515769  5428 tablet_bootstrap.cc:492] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47: No bootstrap required, opened a new log
I20260709 13:14:42.516348  5428 ts_tablet_manager.cc:1403] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47: Time spent bootstrapping tablet: real 0.030s	user 0.025s	sys 0.003s
I20260709 13:14:42.517025  5430 tablet_bootstrap.cc:654] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.518846  5427 raft_consensus.cc:359] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.519637  5427 raft_consensus.cc:385] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.519337  5428 raft_consensus.cc:359] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.519958  5427 raft_consensus.cc:740] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.520255  5428 raft_consensus.cc:385] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.520691  5428 raft_consensus.cc:740] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.520802  5427 consensus_queue.cc:260] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.521831  5428 consensus_queue.cc:260] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.523123  5427 ts_tablet_manager.cc:1434] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:42.524747  5428 ts_tablet_manager.cc:1434] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:14:42.528371  5430 tablet_bootstrap.cc:492] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21: No bootstrap required, opened a new log
I20260709 13:14:42.528887  5430 ts_tablet_manager.cc:1403] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.005s
I20260709 13:14:42.531857  5430 raft_consensus.cc:359] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.532431  5430 raft_consensus.cc:385] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.532676  5430 raft_consensus.cc:740] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd40bda09114c6e92fe9b0e0b0cfe21, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.533318  5430 consensus_queue.cc:260] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.535254  5430 ts_tablet_manager.cc:1434] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:42.586728  5433 raft_consensus.cc:493] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.587322  5433 raft_consensus.cc:515] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.589082  5433 leader_election.cc:290] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.590034  5237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "886a95f2f01b4530a41ae230bac8cf47" is_pre_election: true
I20260709 13:14:42.590238  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" is_pre_election: true
I20260709 13:14:42.590762  5237 raft_consensus.cc:2468] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 0.
I20260709 13:14:42.590799  5386 raft_consensus.cc:2468] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 0.
I20260709 13:14:42.592082  5275 leader_election.cc:304] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 886a95f2f01b4530a41ae230bac8cf47; no voters: 
I20260709 13:14:42.592741  5433 raft_consensus.cc:2804] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:42.593052  5433 raft_consensus.cc:493] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.593320  5433 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.598394  5433 raft_consensus.cc:515] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.600028  5433 leader_election.cc:290] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 1 election: Requested vote from peers 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.600854  5237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
I20260709 13:14:42.601123  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
I20260709 13:14:42.601399  5237 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.601848  5386 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.606446  5237 raft_consensus.cc:2468] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 1.
I20260709 13:14:42.606742  5386 raft_consensus.cc:2468] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 1.
I20260709 13:14:42.607635  5275 leader_election.cc:304] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 886a95f2f01b4530a41ae230bac8cf47; no voters: 
I20260709 13:14:42.608606  5433 raft_consensus.cc:2804] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:42.609140  5433 raft_consensus.cc:697] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Becoming Leader. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Running, Role: LEADER
I20260709 13:14:42.609934  5433 consensus_queue.cc:237] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.616236  5133 catalog_manager.cc:5697] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66). New cstate: current_term: 1 leader_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:42.644940  5434 raft_consensus.cc:493] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.645547  5434 raft_consensus.cc:515] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.647717  5434 leader_election.cc:290] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947), 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827)
I20260709 13:14:42.657931  5435 raft_consensus.cc:493] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.658519  5435 raft_consensus.cc:515] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.659557  5312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" is_pre_election: true
I20260709 13:14:42.659957  5237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "886a95f2f01b4530a41ae230bac8cf47" is_pre_election: true
I20260709 13:14:42.660265  5312 raft_consensus.cc:2468] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd40bda09114c6e92fe9b0e0b0cfe21 in term 0.
I20260709 13:14:42.660640  5435 leader_election.cc:290] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.660812  5237 raft_consensus.cc:2468] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd40bda09114c6e92fe9b0e0b0cfe21 in term 0.
I20260709 13:14:42.661743  5350 leader_election.cc:304] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 5dd40bda09114c6e92fe9b0e0b0cfe21; no voters: 
I20260709 13:14:42.662175  5312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" is_pre_election: true
I20260709 13:14:42.662653  5434 raft_consensus.cc:2804] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:42.662808  5312 raft_consensus.cc:2468] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 0.
I20260709 13:14:42.663187  5434 raft_consensus.cc:493] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.663817  5434 raft_consensus.cc:3060] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.663573  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" is_pre_election: true
I20260709 13:14:42.664301  5201 leader_election.cc:304] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 886a95f2f01b4530a41ae230bac8cf47; no voters: 
I20260709 13:14:42.665063  5435 raft_consensus.cc:2804] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:42.665422  5435 raft_consensus.cc:493] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.665761  5435 raft_consensus.cc:3060] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.671797  5434 raft_consensus.cc:515] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.672983  5386 raft_consensus.cc:2393] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 886a95f2f01b4530a41ae230bac8cf47 in current term 1: Already voted for candidate 5dd40bda09114c6e92fe9b0e0b0cfe21 in this term.
I20260709 13:14:42.673059  5435 raft_consensus.cc:515] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.674165  5434 leader_election.cc:290] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [CANDIDATE]: Term 1 election: Requested vote from peers 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947), 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827)
I20260709 13:14:42.675525  5312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
I20260709 13:14:42.675985  5435 leader_election.cc:290] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 1 election: Requested vote from peers 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.676167  5312 raft_consensus.cc:3060] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.676537  5311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
I20260709 13:14:42.677172  5237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
I20260709 13:14:42.677402  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
I20260709 13:14:42.678081  5237 raft_consensus.cc:2393] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5dd40bda09114c6e92fe9b0e0b0cfe21 in current term 1: Already voted for candidate 886a95f2f01b4530a41ae230bac8cf47 in this term.
I20260709 13:14:42.678262  5386 raft_consensus.cc:2393] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 886a95f2f01b4530a41ae230bac8cf47 in current term 1: Already voted for candidate 5dd40bda09114c6e92fe9b0e0b0cfe21 in this term.
I20260709 13:14:42.679314  5203 leader_election.cc:304] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 886a95f2f01b4530a41ae230bac8cf47; no voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 5dd40bda09114c6e92fe9b0e0b0cfe21
I20260709 13:14:42.679993  5435 raft_consensus.cc:2749] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:42.682790  5312 raft_consensus.cc:2468] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd40bda09114c6e92fe9b0e0b0cfe21 in term 1.
I20260709 13:14:42.683636  5350 leader_election.cc:304] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 5dd40bda09114c6e92fe9b0e0b0cfe21; no voters: 886a95f2f01b4530a41ae230bac8cf47
I20260709 13:14:42.684348  5434 raft_consensus.cc:2804] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:42.685209  5434 raft_consensus.cc:697] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 LEADER]: Becoming Leader. State: Replica: 5dd40bda09114c6e92fe9b0e0b0cfe21, State: Running, Role: LEADER
I20260709 13:14:42.685985  5434 consensus_queue.cc:237] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.692579  5133 catalog_manager.cc:5697] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:42.722544  5312 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cea52ce538b4f08ad59722a3658a9b8"
dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
mode: GRACEFUL
new_leader_uuid: "886a95f2f01b4530a41ae230bac8cf47"
 from {username='slave'} at 127.0.0.1:33214
I20260709 13:14:42.723127  5312 raft_consensus.cc:606] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Received request to transfer leadership to TS 886a95f2f01b4530a41ae230bac8cf47
I20260709 13:14:42.735422  5237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e3bb62c7afc4316ad9100a43f9e5ef2"
dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
mode: GRACEFUL
new_leader_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
 from {username='slave'} at 127.0.0.1:36132
I20260709 13:14:42.735913  5237 raft_consensus.cc:606] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 LEADER]: Received request to transfer leadership to TS 2aaae3c4cdfc4246b9b6e5d878ec3895
I20260709 13:14:42.749606  5386 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033"
dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
mode: GRACEFUL
new_leader_uuid: "886a95f2f01b4530a41ae230bac8cf47"
 from {username='slave'} at 127.0.0.1:49306
I20260709 13:14:42.750182  5386 raft_consensus.cc:606] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 LEADER]: Received request to transfer leadership to TS 886a95f2f01b4530a41ae230bac8cf47
I20260709 13:14:42.755218  5312 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581"
dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
mode: GRACEFUL
new_leader_uuid: "886a95f2f01b4530a41ae230bac8cf47"
 from {username='slave'} at 127.0.0.1:33214
I20260709 13:14:42.755764  5312 raft_consensus.cc:606] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Received request to transfer leadership to TS 886a95f2f01b4530a41ae230bac8cf47
I20260709 13:14:42.767372  5433 consensus_queue.cc:1048] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:42.775686  5445 raft_consensus.cc:993] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895: : Instructing follower 886a95f2f01b4530a41ae230bac8cf47 to start an election
I20260709 13:14:42.776507  5433 raft_consensus.cc:1081] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Signalling peer 886a95f2f01b4530a41ae230bac8cf47 to start an election
I20260709 13:14:42.778755  5236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cea52ce538b4f08ad59722a3658a9b8"
dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
 from {username='slave'} at 127.0.0.1:36116
I20260709 13:14:42.781113  5236 raft_consensus.cc:493] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:42.781536  5236 raft_consensus.cc:3060] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:42.790302  5445 consensus_queue.cc:1048] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:42.791699  5236 raft_consensus.cc:515] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.798503  5312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cea52ce538b4f08ad59722a3658a9b8" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
I20260709 13:14:42.799120  5312 raft_consensus.cc:3055] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:42.799427  5312 raft_consensus.cc:740] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Running, Role: LEADER
I20260709 13:14:42.800763  5312 consensus_queue.cc:260] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.802039  5312 raft_consensus.cc:3060] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:42.803157  5385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cea52ce538b4f08ad59722a3658a9b8" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
I20260709 13:14:42.804153  5236 leader_election.cc:290] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 2 election: Requested vote from peers 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.811108  5312 raft_consensus.cc:2468] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 2.
I20260709 13:14:42.812378  5201 leader_election.cc:304] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 886a95f2f01b4530a41ae230bac8cf47; no voters: 5dd40bda09114c6e92fe9b0e0b0cfe21
I20260709 13:14:42.813161  5435 raft_consensus.cc:2804] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:42.820874  5435 raft_consensus.cc:697] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Becoming Leader. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Running, Role: LEADER
I20260709 13:14:42.821581  5435 consensus_queue.cc:237] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.828029  5132 catalog_manager.cc:5697] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 reported cstate change: term changed from 1 to 2, leader changed from 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66) to 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "886a95f2f01b4530a41ae230bac8cf47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:42.855348  5435 consensus_queue.cc:1048] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:42.863018  5470 raft_consensus.cc:993] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47: : Instructing follower 2aaae3c4cdfc4246b9b6e5d878ec3895 to start an election
I20260709 13:14:42.863756  5435 raft_consensus.cc:1081] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 LEADER]: Signalling peer 2aaae3c4cdfc4246b9b6e5d878ec3895 to start an election
I20260709 13:14:42.864672  5435 consensus_queue.cc:1048] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:42.865513  5311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e3bb62c7afc4316ad9100a43f9e5ef2"
dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
 from {username='slave'} at 127.0.0.1:33186
I20260709 13:14:42.866043  5311 raft_consensus.cc:493] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:42.866420  5311 raft_consensus.cc:3060] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:42.874596  5311 raft_consensus.cc:515] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.877887  5311 leader_election.cc:290] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 2 election: Requested vote from peers 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:42.878979  5236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3bb62c7afc4316ad9100a43f9e5ef2" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
I20260709 13:14:42.879714  5236 raft_consensus.cc:3055] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:42.882120  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3bb62c7afc4316ad9100a43f9e5ef2" candidate_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
I20260709 13:14:42.882200  5236 raft_consensus.cc:740] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Running, Role: LEADER
I20260709 13:14:42.882747  5386 raft_consensus.cc:3060] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:42.883278  5236 consensus_queue.cc:260] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.884714  5236 raft_consensus.cc:3060] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:42.889894  5386 raft_consensus.cc:2468] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 2.
I20260709 13:14:42.891348  5278 leader_election.cc:304] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 5dd40bda09114c6e92fe9b0e0b0cfe21; no voters: 
I20260709 13:14:42.892115  5445 raft_consensus.cc:2804] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:42.892339  5236 raft_consensus.cc:2468] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aaae3c4cdfc4246b9b6e5d878ec3895 in term 2.
I20260709 13:14:42.892547  5445 raft_consensus.cc:697] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 LEADER]: Becoming Leader. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Running, Role: LEADER
I20260709 13:14:42.893498  5445 consensus_queue.cc:237] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:42.900836  5135 catalog_manager.cc:5697] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 reported cstate change: term changed from 1 to 2, leader changed from 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65) to 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66). New cstate: current_term: 2 leader_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:43.005348  5445 consensus_queue.cc:1048] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:43.011420  5433 raft_consensus.cc:993] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895: : Instructing follower 886a95f2f01b4530a41ae230bac8cf47 to start an election
I20260709 13:14:43.012100  5465 raft_consensus.cc:1081] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Signalling peer 886a95f2f01b4530a41ae230bac8cf47 to start an election
I20260709 13:14:43.015445  5237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581"
dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
 from {username='slave'} at 127.0.0.1:36116
I20260709 13:14:43.016064  5237 raft_consensus.cc:493] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:43.016479  5237 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:43.017829  5445 consensus_queue.cc:1048] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:43.024266  5237 raft_consensus.cc:515] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.026607  5237 leader_election.cc:290] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 2 election: Requested vote from peers 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:43.027374  5311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
I20260709 13:14:43.028115  5311 raft_consensus.cc:3055] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:43.028458  5311 raft_consensus.cc:740] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2aaae3c4cdfc4246b9b6e5d878ec3895, State: Running, Role: LEADER
I20260709 13:14:43.029029  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
I20260709 13:14:43.029709  5386 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:43.029275  5311 consensus_queue.cc:260] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.030745  5311 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:43.036438  5386 raft_consensus.cc:2468] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 2.
I20260709 13:14:43.036475  5311 raft_consensus.cc:2468] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 2.
I20260709 13:14:43.037890  5203 leader_election.cc:304] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dd40bda09114c6e92fe9b0e0b0cfe21, 886a95f2f01b4530a41ae230bac8cf47; no voters: 
I20260709 13:14:43.038704  5470 raft_consensus.cc:2804] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:43.039229  5470 raft_consensus.cc:697] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Becoming Leader. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Running, Role: LEADER
I20260709 13:14:43.039981  5470 consensus_queue.cc:237] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.045994  5132 catalog_manager.cc:5697] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 reported cstate change: term changed from 1 to 2, leader changed from 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66) to 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "886a95f2f01b4530a41ae230bac8cf47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:43.156472  5434 consensus_queue.cc:1048] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:43.162649  5461 raft_consensus.cc:993] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21: : Instructing follower 886a95f2f01b4530a41ae230bac8cf47 to start an election
I20260709 13:14:43.163304  5434 raft_consensus.cc:1081] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 LEADER]: Signalling peer 886a95f2f01b4530a41ae230bac8cf47 to start an election
I20260709 13:14:43.165117  5236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033"
dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
 from {username='slave'} at 127.0.0.1:36120
I20260709 13:14:43.165689  5236 raft_consensus.cc:493] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:43.166046  5236 raft_consensus.cc:3060] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:43.168159  5478 consensus_queue.cc:1048] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:43.173200  5236 raft_consensus.cc:515] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.174104  5237 raft_consensus.cc:1240] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Rejecting Update request from peer 5dd40bda09114c6e92fe9b0e0b0cfe21 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:43.176222  5312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
I20260709 13:14:43.177459  5386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03bc9437fb0e4cb1ab2859bed67be033" candidate_uuid: "886a95f2f01b4530a41ae230bac8cf47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
I20260709 13:14:43.177779  5236 leader_election.cc:290] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 2 election: Requested vote from peers 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827), 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67:39263)
I20260709 13:14:43.178126  5386 raft_consensus.cc:3055] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:43.178478  5386 raft_consensus.cc:740] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5dd40bda09114c6e92fe9b0e0b0cfe21, State: Running, Role: LEADER
I20260709 13:14:43.179301  5386 consensus_queue.cc:260] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.180111  5478 consensus_queue.cc:1059] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:43.181212  5386 raft_consensus.cc:3060] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:43.186553  5386 raft_consensus.cc:2468] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 886a95f2f01b4530a41ae230bac8cf47 in term 2.
I20260709 13:14:43.187403  5203 leader_election.cc:304] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5dd40bda09114c6e92fe9b0e0b0cfe21, 886a95f2f01b4530a41ae230bac8cf47; no voters: 2aaae3c4cdfc4246b9b6e5d878ec3895
I20260709 13:14:43.188102  5470 raft_consensus.cc:2804] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:43.188511  5470 raft_consensus.cc:697] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Becoming Leader. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Running, Role: LEADER
I20260709 13:14:43.189257  5470 consensus_queue.cc:237] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.194577  5132 catalog_manager.cc:5697] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 reported cstate change: term changed from 1 to 2, leader changed from 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67) to 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "886a95f2f01b4530a41ae230bac8cf47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:43.214752  5311 raft_consensus.cc:1275] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Refusing update from remote peer 886a95f2f01b4530a41ae230bac8cf47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:43.216104  5470 consensus_queue.cc:1048] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:43.217943 27901 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:14:43.219957 27901 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:14:43.220480 27901 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:14:43.223954  5386 raft_consensus.cc:3060] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:43.233295  5386 raft_consensus.cc:1275] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Refusing update from remote peer 886a95f2f01b4530a41ae230bac8cf47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:43.235320  5470 consensus_queue.cc:1048] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:43.246968  5237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581"
dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
mode: GRACEFUL
new_leader_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
 from {username='slave'} at 127.0.0.1:36146
I20260709 13:14:43.247570  5237 raft_consensus.cc:606] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Received request to transfer leadership to TS 5dd40bda09114c6e92fe9b0e0b0cfe21
I20260709 13:14:43.248757 27901 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:14:43.332063  5385 raft_consensus.cc:1275] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Refusing update from remote peer 2aaae3c4cdfc4246b9b6e5d878ec3895: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:43.333360  5445 consensus_queue.cc:1048] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:43.341534  5237 raft_consensus.cc:1275] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Refusing update from remote peer 2aaae3c4cdfc4246b9b6e5d878ec3895: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:43.343245  5465 consensus_queue.cc:1048] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:43.558480  5311 raft_consensus.cc:1275] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Refusing update from remote peer 886a95f2f01b4530a41ae230bac8cf47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:43.559710  5480 consensus_queue.cc:1048] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:43.569221  5385 raft_consensus.cc:1275] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Refusing update from remote peer 886a95f2f01b4530a41ae230bac8cf47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:43.570518  5473 consensus_queue.cc:1048] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:43.575887  5435 raft_consensus.cc:993] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47: : Instructing follower 5dd40bda09114c6e92fe9b0e0b0cfe21 to start an election
I20260709 13:14:43.576411  5473 raft_consensus.cc:1081] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Signalling peer 5dd40bda09114c6e92fe9b0e0b0cfe21 to start an election
I20260709 13:14:43.577889  5385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581"
dest_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21"
 from {username='slave'} at 127.0.0.1:49300
I20260709 13:14:43.578439  5385 raft_consensus.cc:493] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:43.578820  5385 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:43.584196  5385 raft_consensus.cc:515] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.585774  5385 leader_election.cc:290] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [CANDIDATE]: Term 3 election: Requested vote from peers 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65:36947), 2aaae3c4cdfc4246b9b6e5d878ec3895 (127.27.63.66:33827)
I20260709 13:14:43.586858  5237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581" candidate_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "886a95f2f01b4530a41ae230bac8cf47"
I20260709 13:14:43.586872  5311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc10ad6008dc497aafb97e8e77bc0581" candidate_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895"
I20260709 13:14:43.587946  5237 raft_consensus.cc:3055] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:43.587982  5311 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:43.589704  5237 raft_consensus.cc:740] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 886a95f2f01b4530a41ae230bac8cf47, State: Running, Role: LEADER
I20260709 13:14:43.591357  5237 consensus_queue.cc:260] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.592638  5237 raft_consensus.cc:3060] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:43.596311  5311 raft_consensus.cc:2468] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd40bda09114c6e92fe9b0e0b0cfe21 in term 3.
I20260709 13:14:43.597854  5350 leader_election.cc:304] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aaae3c4cdfc4246b9b6e5d878ec3895, 5dd40bda09114c6e92fe9b0e0b0cfe21; no voters: 
I20260709 13:14:43.597991  5237 raft_consensus.cc:2468] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd40bda09114c6e92fe9b0e0b0cfe21 in term 3.
I20260709 13:14:43.598760  5434 raft_consensus.cc:2804] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:43.599320  5434 raft_consensus.cc:697] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 3 LEADER]: Becoming Leader. State: Replica: 5dd40bda09114c6e92fe9b0e0b0cfe21, State: Running, Role: LEADER
I20260709 13:14:43.600317  5434 consensus_queue.cc:237] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } }
I20260709 13:14:43.605671  5311 raft_consensus.cc:3060] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:43.607084  5135 catalog_manager.cc:5697] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 reported cstate change: term changed from 2 to 3, leader changed from 886a95f2f01b4530a41ae230bac8cf47 (127.27.63.65) to 5dd40bda09114c6e92fe9b0e0b0cfe21 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886a95f2f01b4530a41ae230bac8cf47" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:43.611979  5311 raft_consensus.cc:1275] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Refusing update from remote peer 886a95f2f01b4530a41ae230bac8cf47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:43.613693  5473 consensus_queue.cc:1048] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2aaae3c4cdfc4246b9b6e5d878ec3895" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:43.621673  5385 raft_consensus.cc:1275] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Refusing update from remote peer 886a95f2f01b4530a41ae230bac8cf47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:43.623502  5480 consensus_queue.cc:1048] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd40bda09114c6e92fe9b0e0b0cfe21" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:43.868552 27901 tablet_server.cc:179] TabletServer@127.27.63.65:0 shutting down...
I20260709 13:14:43.892393 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:43.893039 27901 tablet_replica.cc:333] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47: stopping tablet replica
I20260709 13:14:43.893528 27901 raft_consensus.cc:2243] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.893934 27901 raft_consensus.cc:2272] T dc10ad6008dc497aafb97e8e77bc0581 P 886a95f2f01b4530a41ae230bac8cf47 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.896472 27901 tablet_replica.cc:333] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47: stopping tablet replica
I20260709 13:14:43.896939 27901 raft_consensus.cc:2243] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:43.897667 27901 raft_consensus.cc:2272] T 03bc9437fb0e4cb1ab2859bed67be033 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.899726 27901 tablet_replica.cc:333] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47: stopping tablet replica
I20260709 13:14:43.900241 27901 raft_consensus.cc:2243] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:43.900924 27901 raft_consensus.cc:2272] T 2cea52ce538b4f08ad59722a3658a9b8 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.902856 27901 tablet_replica.cc:333] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47: stopping tablet replica
I20260709 13:14:43.903275 27901 raft_consensus.cc:2243] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.903627 27901 raft_consensus.cc:2272] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 886a95f2f01b4530a41ae230bac8cf47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.927453 27901 tablet_server.cc:196] TabletServer@127.27.63.65:0 shutdown complete.
I20260709 13:14:43.940780 27901 tablet_server.cc:179] TabletServer@127.27.63.66:0 shutting down...
I20260709 13:14:43.961304 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:43.961917 27901 tablet_replica.cc:333] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895: stopping tablet replica
I20260709 13:14:43.962415 27901 raft_consensus.cc:2243] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.962805 27901 raft_consensus.cc:2272] T 03bc9437fb0e4cb1ab2859bed67be033 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.965268 27901 tablet_replica.cc:333] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895: stopping tablet replica
I20260709 13:14:43.965728 27901 raft_consensus.cc:2243] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.966091 27901 raft_consensus.cc:2272] T dc10ad6008dc497aafb97e8e77bc0581 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.968384 27901 tablet_replica.cc:333] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895: stopping tablet replica
I20260709 13:14:43.968832 27901 raft_consensus.cc:2243] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.969209 27901 raft_consensus.cc:2272] T 2cea52ce538b4f08ad59722a3658a9b8 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.971218 27901 tablet_replica.cc:333] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895: stopping tablet replica
I20260709 13:14:43.971643 27901 raft_consensus.cc:2243] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:43.972410 27901 raft_consensus.cc:2272] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 2aaae3c4cdfc4246b9b6e5d878ec3895 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.985245 27901 tablet_server.cc:196] TabletServer@127.27.63.66:0 shutdown complete.
I20260709 13:14:43.997634 27901 tablet_server.cc:179] TabletServer@127.27.63.67:0 shutting down...
I20260709 13:14:44.018033 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:44.018788 27901 tablet_replica.cc:333] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21: stopping tablet replica
I20260709 13:14:44.019368 27901 raft_consensus.cc:2243] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:44.019847 27901 raft_consensus.cc:2272] T 03bc9437fb0e4cb1ab2859bed67be033 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.021932 27901 tablet_replica.cc:333] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21: stopping tablet replica
I20260709 13:14:44.022455 27901 raft_consensus.cc:2243] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:44.023069 27901 pending_rounds.cc:70] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Trying to abort 1 pending ops.
I20260709 13:14:44.023272 27901 pending_rounds.cc:77] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305637411236151296 op_type: NO_OP noop_request { }
I20260709 13:14:44.023572 27901 raft_consensus.cc:2889] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:44.023840 27901 raft_consensus.cc:2272] T dc10ad6008dc497aafb97e8e77bc0581 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.025972 27901 tablet_replica.cc:333] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21: stopping tablet replica
I20260709 13:14:44.026453 27901 raft_consensus.cc:2243] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:44.026868 27901 raft_consensus.cc:2272] T 8e3bb62c7afc4316ad9100a43f9e5ef2 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.029102 27901 tablet_replica.cc:333] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21: stopping tablet replica
I20260709 13:14:44.029562 27901 raft_consensus.cc:2243] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:44.029960 27901 raft_consensus.cc:2272] T 2cea52ce538b4f08ad59722a3658a9b8 P 5dd40bda09114c6e92fe9b0e0b0cfe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.041895 27901 tablet_server.cc:196] TabletServer@127.27.63.67:0 shutdown complete.
I20260709 13:14:44.055063 27901 master.cc:562] Master@127.27.63.126:46141 shutting down...
I20260709 13:14:44.073808 27901 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:44.074369 27901 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.074716 27901 tablet_replica.cc:333] T 00000000000000000000000000000000 P 37c6f9655e464423b1635630bd3a09d2: stopping tablet replica
I20260709 13:14:44.093398 27901 master.cc:584] Master@127.27.63.126:46141 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2806 ms)
[----------] 6 tests from LeaderRebalancerTest (292350 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:14:44.128566 27901 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.63.126:46611
I20260709 13:14:44.129767 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:44.135179  5482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:44.135367  5483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:44.136467  5485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:44.137367 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:44.138165 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:44.138358 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:44.138536 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602884138488 us; error 0 us; skew 500 ppm
I20260709 13:14:44.139056 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:44.141498 27901 webserver.cc:533] Webserver started at http://127.27.63.126:46821/ using document root <none> and password file <none>
I20260709 13:14:44.142012 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:44.142216 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:44.142493 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:44.144004 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/master-0-root/instance:
uuid: "2cf7de9bb6ed4420b26939d4cde7325b"
format_stamp: "Formatted at 2026-07-09 13:14:44 on dist-test-slave-sjgx"
I20260709 13:14:44.148439 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:44.151600  5490 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.152383 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:44.152681 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/master-0-root
uuid: "2cf7de9bb6ed4420b26939d4cde7325b"
format_stamp: "Formatted at 2026-07-09 13:14:44 on dist-test-slave-sjgx"
I20260709 13:14:44.152966 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:44.167933 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:44.169147 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:44.206662 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.126:46611
I20260709 13:14:44.206782  5541 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.126:46611 every 8 connection(s)
I20260709 13:14:44.210441  5542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.220986  5542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b: Bootstrap starting.
I20260709 13:14:44.225682  5542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.229779  5542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b: No bootstrap required, opened a new log
I20260709 13:14:44.231818  5542 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf7de9bb6ed4420b26939d4cde7325b" member_type: VOTER }
I20260709 13:14:44.232285  5542 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.232514  5542 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cf7de9bb6ed4420b26939d4cde7325b, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.233161  5542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf7de9bb6ed4420b26939d4cde7325b" member_type: VOTER }
I20260709 13:14:44.233631  5542 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:44.233856  5542 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:44.234127  5542 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.238796  5542 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf7de9bb6ed4420b26939d4cde7325b" member_type: VOTER }
I20260709 13:14:44.239310  5542 leader_election.cc:304] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cf7de9bb6ed4420b26939d4cde7325b; no voters: 
I20260709 13:14:44.240451  5542 leader_election.cc:290] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:44.240808  5545 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:44.242133  5545 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 1 LEADER]: Becoming Leader. State: Replica: 2cf7de9bb6ed4420b26939d4cde7325b, State: Running, Role: LEADER
I20260709 13:14:44.242825  5545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf7de9bb6ed4420b26939d4cde7325b" member_type: VOTER }
I20260709 13:14:44.243148  5542 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:44.245854  5547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cf7de9bb6ed4420b26939d4cde7325b. Latest consensus state: current_term: 1 leader_uuid: "2cf7de9bb6ed4420b26939d4cde7325b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf7de9bb6ed4420b26939d4cde7325b" member_type: VOTER } }
I20260709 13:14:44.245769  5546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cf7de9bb6ed4420b26939d4cde7325b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf7de9bb6ed4420b26939d4cde7325b" member_type: VOTER } }
I20260709 13:14:44.246549  5547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:44.246867  5546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:44.250830  5551 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:44.256150  5551 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:44.259126 27901 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:44.264391  5551 catalog_manager.cc:1383] Generated new cluster ID: 5c993202cb42438f97a11542a6e4901e
I20260709 13:14:44.264694  5551 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:44.276890  5551 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:44.278180  5551 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:44.294919  5551 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b: Generated new TSK 0
I20260709 13:14:44.295531  5551 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:44.326059 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:44.331715  5563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:44.332875  5564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:44.334355  5566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:44.334645 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:44.335599 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:44.335806 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:44.335979 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602884335960 us; error 0 us; skew 500 ppm
I20260709 13:14:44.336598 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:44.338748 27901 webserver.cc:533] Webserver started at http://127.27.63.65:32797/ using document root <none> and password file <none>
I20260709 13:14:44.339247 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:44.339442 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:44.339733 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:44.340868 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-0-root/instance:
uuid: "03fb445d9b604085af1649bdedf5f55e"
format_stamp: "Formatted at 2026-07-09 13:14:44 on dist-test-slave-sjgx"
I20260709 13:14:44.345400 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:14:44.348747  5571 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.349419 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:44.349754 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "03fb445d9b604085af1649bdedf5f55e"
format_stamp: "Formatted at 2026-07-09 13:14:44 on dist-test-slave-sjgx"
I20260709 13:14:44.350174 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:44.367131 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:44.369107 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:44.371356 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:44.374975 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:44.375274 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.375590 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:44.375779 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.419621 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:46869
I20260709 13:14:44.419713  5633 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:46869 every 8 connection(s)
I20260709 13:14:44.426182 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:44.434089  5638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:44.434150  5639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:44.438606  5641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:44.442404 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:44.443269  5634 heartbeater.cc:344] Connected to a master server at 127.27.63.126:46611
I20260709 13:14:44.443742  5634 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:44.444315 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:44.444633 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:44.444674  5634 heartbeater.cc:507] Master 127.27.63.126:46611 requested a full tablet report, sending...
I20260709 13:14:44.444906 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602884444882 us; error 0 us; skew 500 ppm
I20260709 13:14:44.445740 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:44.447150  5507 ts_manager.cc:194] Registered new tserver with Master: 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869)
I20260709 13:14:44.448313 27901 webserver.cc:533] Webserver started at http://127.27.63.66:38019/ using document root <none> and password file <none>
I20260709 13:14:44.448835 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:44.449043 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:44.449276  5507 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47578
I20260709 13:14:44.449304 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:44.450682 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-1-root/instance:
uuid: "204d6918cdaf414f9860e5cbc718b0aa"
format_stamp: "Formatted at 2026-07-09 13:14:44 on dist-test-slave-sjgx"
I20260709 13:14:44.455245 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:14:44.458388  5646 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.459165 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:14:44.459404 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-1-root
uuid: "204d6918cdaf414f9860e5cbc718b0aa"
format_stamp: "Formatted at 2026-07-09 13:14:44 on dist-test-slave-sjgx"
I20260709 13:14:44.459635 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:44.474737 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:44.475710 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:44.477087 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:44.479178 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:44.479377 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.479624 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:44.479780 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.515731 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.66:36993
I20260709 13:14:44.515836  5708 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.66:36993 every 8 connection(s)
I20260709 13:14:44.520483 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:44.526567  5712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:44.527537  5713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:44.530372  5715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:44.531451 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:44.531865  5709 heartbeater.cc:344] Connected to a master server at 127.27.63.126:46611
I20260709 13:14:44.532358  5709 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:44.532653 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:44.532948 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:44.533167 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602884533150 us; error 0 us; skew 500 ppm
I20260709 13:14:44.533255  5709 heartbeater.cc:507] Master 127.27.63.126:46611 requested a full tablet report, sending...
I20260709 13:14:44.533948 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:44.535359  5507 ts_manager.cc:194] Registered new tserver with Master: 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:44.536744 27901 webserver.cc:533] Webserver started at http://127.27.63.67:33375/ using document root <none> and password file <none>
I20260709 13:14:44.536978  5507 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47584
I20260709 13:14:44.537307 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:44.537572 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:44.537850 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:44.539022 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-2-root/instance:
uuid: "61b1acaac8bf494cb41df91d5c6affca"
format_stamp: "Formatted at 2026-07-09 13:14:44 on dist-test-slave-sjgx"
I20260709 13:14:44.543184 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 13:14:44.546293  5720 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.546959 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.001s
I20260709 13:14:44.547251 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-2-root
uuid: "61b1acaac8bf494cb41df91d5c6affca"
format_stamp: "Formatted at 2026-07-09 13:14:44 on dist-test-slave-sjgx"
I20260709 13:14:44.547534 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602591602838-27901-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:44.557978 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:44.558969 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:44.560403 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:44.562585 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:44.562758 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.563045 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:44.563203 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:44.602821 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.67:36081
I20260709 13:14:44.602932  5782 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.67:36081 every 8 connection(s)
I20260709 13:14:44.616299  5783 heartbeater.cc:344] Connected to a master server at 127.27.63.126:46611
I20260709 13:14:44.616771  5783 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:44.617501  5783 heartbeater.cc:507] Master 127.27.63.126:46611 requested a full tablet report, sending...
I20260709 13:14:44.619483  5507 ts_manager.cc:194] Registered new tserver with Master: 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:44.619652 27901 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013686762s
I20260709 13:14:44.621308  5507 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47588
I20260709 13:14:44.643712  5507 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47604:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:44.646242  5507 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:44.708948  5674 tablet_service.cc:1511] Processing CreateTablet for tablet 03158be0aa71421d89031e4b7fad6af2 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:44.709209  5673 tablet_service.cc:1511] Processing CreateTablet for tablet 9f59faadac8a40caad4aead074d3ad1f (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:44.710421  5674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03158be0aa71421d89031e4b7fad6af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.711126  5673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f59faadac8a40caad4aead074d3ad1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.714563  5672 tablet_service.cc:1511] Processing CreateTablet for tablet 924c6b951ad34e9f9cc3610764e2f7c3 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:44.716056  5672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924c6b951ad34e9f9cc3610764e2f7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.721526  5671 tablet_service.cc:1511] Processing CreateTablet for tablet 09345849992a47948144e243e591e4ea (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:44.722884  5671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09345849992a47948144e243e591e4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.729609  5670 tablet_service.cc:1511] Processing CreateTablet for tablet 5284f1c6b1f24d938f03bebbdf7b3bde (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:44.731070  5670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5284f1c6b1f24d938f03bebbdf7b3bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.733530  5669 tablet_service.cc:1511] Processing CreateTablet for tablet 91ba4865f65a49a093a45bacdee5f8cd (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:44.741371  5668 tablet_service.cc:1511] Processing CreateTablet for tablet 1233b88a15c64dea8fea255c68f3b264 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:44.743053  5669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ba4865f65a49a093a45bacdee5f8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.742916  5746 tablet_service.cc:1511] Processing CreateTablet for tablet 924c6b951ad34e9f9cc3610764e2f7c3 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:44.744372  5668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1233b88a15c64dea8fea255c68f3b264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.745455  5745 tablet_service.cc:1511] Processing CreateTablet for tablet 09345849992a47948144e243e591e4ea (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:44.748055  5744 tablet_service.cc:1511] Processing CreateTablet for tablet 5284f1c6b1f24d938f03bebbdf7b3bde (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:44.749523  5667 tablet_service.cc:1511] Processing CreateTablet for tablet 5bfad76b6e004e95bc768e524eaa3b62 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:44.750957  5667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bfad76b6e004e95bc768e524eaa3b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.752691  5748 tablet_service.cc:1511] Processing CreateTablet for tablet 03158be0aa71421d89031e4b7fad6af2 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:44.753672  5666 tablet_service.cc:1511] Processing CreateTablet for tablet 59bd46f0367d4b96ab704bb1a32ab93d (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:44.750590  5747 tablet_service.cc:1511] Processing CreateTablet for tablet 9f59faadac8a40caad4aead074d3ad1f (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:44.755367  5743 tablet_service.cc:1511] Processing CreateTablet for tablet 91ba4865f65a49a093a45bacdee5f8cd (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:44.756386  5666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59bd46f0367d4b96ab704bb1a32ab93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.757053  5746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924c6b951ad34e9f9cc3610764e2f7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.761531  5599 tablet_service.cc:1511] Processing CreateTablet for tablet 03158be0aa71421d89031e4b7fad6af2 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:44.762902  5599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03158be0aa71421d89031e4b7fad6af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.764695  5598 tablet_service.cc:1511] Processing CreateTablet for tablet 9f59faadac8a40caad4aead074d3ad1f (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:44.766004  5598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f59faadac8a40caad4aead074d3ad1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.766947  5742 tablet_service.cc:1511] Processing CreateTablet for tablet 1233b88a15c64dea8fea255c68f3b264 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:44.769116  5597 tablet_service.cc:1511] Processing CreateTablet for tablet 924c6b951ad34e9f9cc3610764e2f7c3 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:44.769527  5741 tablet_service.cc:1511] Processing CreateTablet for tablet 5bfad76b6e004e95bc768e524eaa3b62 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:44.770409  5597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924c6b951ad34e9f9cc3610764e2f7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.775291  5596 tablet_service.cc:1511] Processing CreateTablet for tablet 09345849992a47948144e243e591e4ea (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:44.776681  5596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09345849992a47948144e243e591e4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.781212  5595 tablet_service.cc:1511] Processing CreateTablet for tablet 5284f1c6b1f24d938f03bebbdf7b3bde (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:44.782658  5595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5284f1c6b1f24d938f03bebbdf7b3bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.772007  5740 tablet_service.cc:1511] Processing CreateTablet for tablet 59bd46f0367d4b96ab704bb1a32ab93d (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:44.785127  5594 tablet_service.cc:1511] Processing CreateTablet for tablet 91ba4865f65a49a093a45bacdee5f8cd (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:44.786479  5594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ba4865f65a49a093a45bacdee5f8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.789296  5593 tablet_service.cc:1511] Processing CreateTablet for tablet 1233b88a15c64dea8fea255c68f3b264 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:44.790683  5593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1233b88a15c64dea8fea255c68f3b264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.793725  5592 tablet_service.cc:1511] Processing CreateTablet for tablet 5bfad76b6e004e95bc768e524eaa3b62 (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:44.795085  5592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bfad76b6e004e95bc768e524eaa3b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.797622  5591 tablet_service.cc:1511] Processing CreateTablet for tablet 59bd46f0367d4b96ab704bb1a32ab93d (DEFAULT_TABLE table=test-workload [id=fda3ef665e9a478ea1e5a8135be24e90]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:44.763543  5745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09345849992a47948144e243e591e4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.799590  5591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59bd46f0367d4b96ab704bb1a32ab93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.800688  5740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59bd46f0367d4b96ab704bb1a32ab93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.801297  5741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bfad76b6e004e95bc768e524eaa3b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.804353  5742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1233b88a15c64dea8fea255c68f3b264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.806388  5748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03158be0aa71421d89031e4b7fad6af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.807770  5747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f59faadac8a40caad4aead074d3ad1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.810389  5744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5284f1c6b1f24d938f03bebbdf7b3bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.812327  5743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ba4865f65a49a093a45bacdee5f8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.856122  5797 tablet_bootstrap.cc:492] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:44.873332  5798 tablet_bootstrap.cc:492] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:44.876552  5797 tablet_bootstrap.cc:654] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.892940  5798 tablet_bootstrap.cc:654] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.909570  5799 tablet_bootstrap.cc:492] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:44.910271  5797 tablet_bootstrap.cc:492] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:44.910637  5798 tablet_bootstrap.cc:492] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:44.910799  5797 ts_tablet_manager.cc:1403] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.007s
I20260709 13:14:44.911114  5798 ts_tablet_manager.cc:1403] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.004s
I20260709 13:14:44.913784  5798 raft_consensus.cc:359] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.914040  5797 raft_consensus.cc:359] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.914505  5798 raft_consensus.cc:385] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.914855  5797 raft_consensus.cc:385] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.915023  5798 raft_consensus.cc:740] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.915237  5797 raft_consensus.cc:740] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.915910  5798 consensus_queue.cc:260] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.916356  5799 tablet_bootstrap.cc:654] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.916074  5797 consensus_queue.cc:260] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.918078  5634 heartbeater.cc:499] Master 127.27.63.126:46611 was elected leader, sending a full tablet report...
I20260709 13:14:44.918602  5798 ts_tablet_manager.cc:1434] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:44.919561  5798 tablet_bootstrap.cc:492] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:44.926285  5798 tablet_bootstrap.cc:654] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.927145  5797 ts_tablet_manager.cc:1434] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.016s	user 0.012s	sys 0.004s
I20260709 13:14:44.927820  5799 tablet_bootstrap.cc:492] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:44.928301  5799 ts_tablet_manager.cc:1403] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.005s
I20260709 13:14:44.930135  5709 heartbeater.cc:499] Master 127.27.63.126:46611 was elected leader, sending a full tablet report...
I20260709 13:14:44.935865  5797 tablet_bootstrap.cc:492] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:44.937528  5798 tablet_bootstrap.cc:492] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:44.938107  5798 ts_tablet_manager.cc:1403] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 13:14:44.941154  5798 raft_consensus.cc:359] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.941933  5798 raft_consensus.cc:385] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.942298  5798 raft_consensus.cc:740] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.942893  5797 tablet_bootstrap.cc:654] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.943118  5798 consensus_queue.cc:260] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.945927  5798 ts_tablet_manager.cc:1434] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:44.946873  5798 tablet_bootstrap.cc:492] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:44.949024  5799 raft_consensus.cc:359] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.950217  5799 raft_consensus.cc:385] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.950702  5799 raft_consensus.cc:740] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.951890  5799 consensus_queue.cc:260] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.953469  5798 tablet_bootstrap.cc:654] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.956245  5783 heartbeater.cc:499] Master 127.27.63.126:46611 was elected leader, sending a full tablet report...
I20260709 13:14:44.957118  5799 ts_tablet_manager.cc:1434] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:14:44.958189  5799 tablet_bootstrap.cc:492] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:44.962364  5797 tablet_bootstrap.cc:492] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:44.962903  5797 ts_tablet_manager.cc:1403] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.004s
I20260709 13:14:44.964885  5799 tablet_bootstrap.cc:654] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.965845  5797 raft_consensus.cc:359] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.966611  5797 raft_consensus.cc:385] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.966902  5797 raft_consensus.cc:740] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.967826  5797 consensus_queue.cc:260] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.971081  5797 ts_tablet_manager.cc:1434] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:14:44.972334  5797 tablet_bootstrap.cc:492] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:44.980734  5797 tablet_bootstrap.cc:654] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.993170  5797 tablet_bootstrap.cc:492] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:44.993397  5799 tablet_bootstrap.cc:492] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:44.993752  5797 ts_tablet_manager.cc:1403] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 13:14:44.993937  5799 ts_tablet_manager.cc:1403] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.017s
I20260709 13:14:44.995038  5798 tablet_bootstrap.cc:492] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:44.995620  5798 ts_tablet_manager.cc:1403] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.029s
I20260709 13:14:44.996815  5797 raft_consensus.cc:359] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.997025  5799 raft_consensus.cc:359] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.997745  5797 raft_consensus.cc:385] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.997889  5799 raft_consensus.cc:385] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.998138  5797 raft_consensus.cc:740] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.998317  5799 raft_consensus.cc:740] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.999127  5798 raft_consensus.cc:359] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.999219  5797 consensus_queue.cc:260] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:44.999344  5799 consensus_queue.cc:260] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.000022  5798 raft_consensus.cc:385] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.000579  5798 raft_consensus.cc:740] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.001720  5797 ts_tablet_manager.cc:1434] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:14:45.001497  5798 consensus_queue.cc:260] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.002277  5799 ts_tablet_manager.cc:1434] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:14:45.002753  5797 tablet_bootstrap.cc:492] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:45.003264  5799 tablet_bootstrap.cc:492] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:45.004299  5798 ts_tablet_manager.cc:1434] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:14:45.005371  5798 tablet_bootstrap.cc:492] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:45.009289  5797 tablet_bootstrap.cc:654] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.009824  5799 tablet_bootstrap.cc:654] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.011159  5798 tablet_bootstrap.cc:654] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.021384  5797 tablet_bootstrap.cc:492] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:45.021492  5799 tablet_bootstrap.cc:492] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:45.021938  5797 ts_tablet_manager.cc:1403] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:14:45.022116  5799 ts_tablet_manager.cc:1403] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:14:45.021522  5798 tablet_bootstrap.cc:492] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:45.024580  5798 ts_tablet_manager.cc:1403] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.005s
I20260709 13:14:45.025121  5797 raft_consensus.cc:359] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.025405  5799 raft_consensus.cc:359] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.026217  5799 raft_consensus.cc:385] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.026535  5799 raft_consensus.cc:740] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.027323  5799 consensus_queue.cc:260] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.027946  5798 raft_consensus.cc:359] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.028806  5798 raft_consensus.cc:385] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.025898  5797 raft_consensus.cc:385] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.029156  5798 raft_consensus.cc:740] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.029400  5797 raft_consensus.cc:740] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.029896  5799 ts_tablet_manager.cc:1434] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:45.031013  5799 tablet_bootstrap.cc:492] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:45.030365  5797 consensus_queue.cc:260] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.030360  5798 consensus_queue.cc:260] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.035597  5798 ts_tablet_manager.cc:1434] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:45.035830  5797 ts_tablet_manager.cc:1434] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:14:45.036796  5798 tablet_bootstrap.cc:492] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:45.037930  5799 tablet_bootstrap.cc:654] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.036852  5797 tablet_bootstrap.cc:492] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:45.046628  5797 tablet_bootstrap.cc:654] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.050895  5799 tablet_bootstrap.cc:492] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:45.051388  5799 ts_tablet_manager.cc:1403] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.003s
I20260709 13:14:45.052894  5798 tablet_bootstrap.cc:654] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.053843  5797 tablet_bootstrap.cc:492] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:45.054260  5802 raft_consensus.cc:493] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.054417  5797 ts_tablet_manager.cc:1403] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 13:14:45.054862  5802 raft_consensus.cc:515] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.054319  5799 raft_consensus.cc:359] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.055780  5799 raft_consensus.cc:385] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.056166  5799 raft_consensus.cc:740] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.057106  5799 consensus_queue.cc:260] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.057686  5802 leader_election.cc:290] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.060360  5799 ts_tablet_manager.cc:1434] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:14:45.061936  5799 tablet_bootstrap.cc:492] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:45.070739  5799 tablet_bootstrap.cc:654] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.076705  5797 raft_consensus.cc:359] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.077831  5797 raft_consensus.cc:385] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.078210  5797 raft_consensus.cc:740] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.079208  5797 consensus_queue.cc:260] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.082193  5797 ts_tablet_manager.cc:1434] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.004s
I20260709 13:14:45.083287  5797 tablet_bootstrap.cc:492] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:45.095160  5797 tablet_bootstrap.cc:654] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.098773  5807 raft_consensus.cc:493] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.099181  5798 tablet_bootstrap.cc:492] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:45.099759  5798 ts_tablet_manager.cc:1403] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.022s
I20260709 13:14:45.099377  5807 raft_consensus.cc:515] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.101663  5799 tablet_bootstrap.cc:492] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:45.102342  5799 ts_tablet_manager.cc:1403] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.016s
I20260709 13:14:45.102890  5798 raft_consensus.cc:359] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.103665  5798 raft_consensus.cc:385] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.103950  5798 raft_consensus.cc:740] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.104727  5798 consensus_queue.cc:260] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.106344  5799 raft_consensus.cc:359] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.107357  5799 raft_consensus.cc:385] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.107751  5799 raft_consensus.cc:740] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.108774  5799 consensus_queue.cc:260] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.115345  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924c6b951ad34e9f9cc3610764e2f7c3" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:45.115799  5798 ts_tablet_manager.cc:1434] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:14:45.116216  5609 raft_consensus.cc:2468] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:45.117754  5721 leader_election.cc:304] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.121876  5807 leader_election.cc:290] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.123744  5798 tablet_bootstrap.cc:492] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:45.125562  5807 raft_consensus.cc:2804] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.125994  5807 raft_consensus.cc:493] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.126317  5807 raft_consensus.cc:3060] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.132170  5798 tablet_bootstrap.cc:654] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.156400  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
W20260709 13:14:45.158617  5572 leader_election.cc:343] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.161949  5803 raft_consensus.cc:493] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.162725  5803 raft_consensus.cc:515] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.168648  5803 leader_election.cc:290] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
W20260709 13:14:45.193862  5721 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:46869, user_credentials={real_user=slave}} blocked reactor thread for 76363.8us
I20260709 13:14:45.197641  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.198714  5684 raft_consensus.cc:2468] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 0.
I20260709 13:14:45.200662  5575 leader_election.cc:304] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:45.202253  5802 raft_consensus.cc:2804] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.202747  5802 raft_consensus.cc:493] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.203183  5802 raft_consensus.cc:3060] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.218849  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924c6b951ad34e9f9cc3610764e2f7c3" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.219650  5684 raft_consensus.cc:2468] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:45.222164  5802 raft_consensus.cc:515] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.226225  5799 ts_tablet_manager.cc:1434] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.123s	user 0.060s	sys 0.059s
I20260709 13:14:45.229475  5799 tablet_bootstrap.cc:492] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:45.233652  5807 raft_consensus.cc:515] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.237052  5799 tablet_bootstrap.cc:654] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.236852  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924c6b951ad34e9f9cc3610764e2f7c3" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:45.237717  5609 raft_consensus.cc:3060] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.249591  5799 tablet_bootstrap.cc:492] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:45.250186  5799 ts_tablet_manager.cc:1403] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:14:45.250592  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:45.252624  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:45.252897  5802 leader_election.cc:290] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.253551  5799 raft_consensus.cc:359] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
W20260709 13:14:45.254109  5555 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:45.254388  5799 raft_consensus.cc:385] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:14:45.254590  5572 leader_election.cc:343] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.254709  5799 raft_consensus.cc:740] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
W20260709 13:14:45.254153  5647 leader_election.cc:343] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.256126  5608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:45.255934  5799 consensus_queue.cc:260] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.257268  5608 raft_consensus.cc:2393] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 204d6918cdaf414f9860e5cbc718b0aa in current term 1: Already voted for candidate 03fb445d9b604085af1649bdedf5f55e in this term.
I20260709 13:14:45.257871  5609 raft_consensus.cc:2468] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:45.259179  5647 leader_election.cc:304] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa; no voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:45.259541  5721 leader_election.cc:304] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.260545  5803 raft_consensus.cc:3060] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.263901  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:45.253528  5802 raft_consensus.cc:493] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.266254  5802 raft_consensus.cc:515] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.267994  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924c6b951ad34e9f9cc3610764e2f7c3" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:45.268203  5807 leader_election.cc:290] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.268699  5683 raft_consensus.cc:3060] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.268059  5799 ts_tablet_manager.cc:1434] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:14:45.270066  5821 raft_consensus.cc:2804] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.270220  5799 tablet_bootstrap.cc:492] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:45.270642  5821 raft_consensus.cc:697] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:45.271530  5821 consensus_queue.cc:237] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.274634  5803 raft_consensus.cc:2749] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.276072  5683 raft_consensus.cc:2468] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:45.279412  5799 tablet_bootstrap.cc:654] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.279475  5798 tablet_bootstrap.cc:492] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:45.281155  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09345849992a47948144e243e591e4ea" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.281488  5798 ts_tablet_manager.cc:1403] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.158s	user 0.057s	sys 0.070s
I20260709 13:14:45.281842  5683 raft_consensus.cc:2468] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 0.
I20260709 13:14:45.283087  5575 leader_election.cc:304] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:45.284986  5802 leader_election.cc:290] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.285190  5684 raft_consensus.cc:2468] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 1.
I20260709 13:14:45.285799  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09345849992a47948144e243e591e4ea" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:45.286259  5798 raft_consensus.cc:359] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.287180  5797 tablet_bootstrap.cc:492] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:45.286681  5758 raft_consensus.cc:2468] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 0.
I20260709 13:14:45.287272  5798 raft_consensus.cc:385] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.287842  5797 ts_tablet_manager.cc:1403] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.205s	user 0.032s	sys 0.066s
I20260709 13:14:45.287777  5798 raft_consensus.cc:740] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.288803  5798 consensus_queue.cc:260] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.289572  5575 leader_election.cc:304] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:45.284526  5828 raft_consensus.cc:2804] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.291232  5828 raft_consensus.cc:493] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.290982  5797 raft_consensus.cc:359] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.291823  5797 raft_consensus.cc:385] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.291738  5828 raft_consensus.cc:3060] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.292109  5802 raft_consensus.cc:2804] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.292366  5797 raft_consensus.cc:740] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.293315  5797 consensus_queue.cc:260] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.295377  5798 ts_tablet_manager.cc:1434] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:14:45.295665  5797 ts_tablet_manager.cc:1434] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:45.296269  5807 raft_consensus.cc:493] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.296490  5798 tablet_bootstrap.cc:492] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:45.296782  5807 raft_consensus.cc:515] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.301291  5797 tablet_bootstrap.cc:492] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:45.301457  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:45.301599  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
W20260709 13:14:45.303294  5721 leader_election.cc:343] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.303910  5684 raft_consensus.cc:2468] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:45.305367  5724 leader_election.cc:304] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:45.306293  5821 raft_consensus.cc:2804] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.306638  5821 raft_consensus.cc:493] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.306980  5821 raft_consensus.cc:3060] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.308444  5797 tablet_bootstrap.cc:654] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.315001  5821 raft_consensus.cc:515] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.316493  5506 catalog_manager.cc:5697] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.317826  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
W20260709 13:14:45.319581  5721 leader_election.cc:343] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.320432  5807 leader_election.cc:290] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.322067  5802 raft_consensus.cc:697] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:45.322868  5803 raft_consensus.cc:493] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.323042  5802 consensus_queue.cc:237] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.323439  5803 raft_consensus.cc:515] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.325524  5803 leader_election.cc:290] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.327268  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ba4865f65a49a093a45bacdee5f8cd" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
W20260709 13:14:45.329020  5647 leader_election.cc:343] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.329114  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ba4865f65a49a093a45bacdee5f8cd" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:45.323398  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:45.335431  5799 tablet_bootstrap.cc:492] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:45.330165  5758 raft_consensus.cc:2468] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 0.
I20260709 13:14:45.336611  5799 ts_tablet_manager.cc:1403] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.067s	user 0.017s	sys 0.010s
I20260709 13:14:45.337110  5684 raft_consensus.cc:3060] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.337646  5647 leader_election.cc:304] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:45.338922  5803 raft_consensus.cc:2804] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.339310  5803 raft_consensus.cc:493] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.339649  5803 raft_consensus.cc:3060] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.340422  5799 raft_consensus.cc:359] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.341339  5799 raft_consensus.cc:385] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.340852  5828 raft_consensus.cc:515] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.341723  5799 raft_consensus.cc:740] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.342906  5799 consensus_queue.cc:260] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.345247  5828 leader_election.cc:290] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.330382  5807 raft_consensus.cc:493] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.345300  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09345849992a47948144e243e591e4ea" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:45.346496  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09345849992a47948144e243e591e4ea" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:45.346776  5683 raft_consensus.cc:3060] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.347091  5758 raft_consensus.cc:3060] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.346715  5807 raft_consensus.cc:515] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.330263  5821 leader_election.cc:290] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.351948  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bd46f0367d4b96ab704bb1a32ab93d" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:45.352662  5609 raft_consensus.cc:2468] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:45.354115  5721 leader_election.cc:304] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.357446  5683 raft_consensus.cc:2468] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 1.
I20260709 13:14:45.358114  5821 raft_consensus.cc:2804] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.359534  5821 raft_consensus.cc:493] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.358701  5758 raft_consensus.cc:2468] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 1.
I20260709 13:14:45.346104  5799 ts_tablet_manager.cc:1434] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:14:45.359786  5803 raft_consensus.cc:515] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.361265  5799 tablet_bootstrap.cc:492] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:45.363531  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ba4865f65a49a093a45bacdee5f8cd" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:45.359944  5821 raft_consensus.cc:3060] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:45.365307  5647 leader_election.cc:343] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.367761  5757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ba4865f65a49a093a45bacdee5f8cd" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:45.368430  5757 raft_consensus.cc:3060] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.371062  5803 leader_election.cc:290] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.371997  5575 leader_election.cc:304] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:45.372774  5802 raft_consensus.cc:2804] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.372797  5798 tablet_bootstrap.cc:654] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.373308  5802 raft_consensus.cc:697] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:45.374264  5802 consensus_queue.cc:237] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.380348  5799 tablet_bootstrap.cc:654] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.382959  5684 raft_consensus.cc:2468] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:45.384766  5807 leader_election.cc:290] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.385154  5505 catalog_manager.cc:5697] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 0 to 1, leader changed from <none> to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 1 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.386236  5797 tablet_bootstrap.cc:492] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:45.386672  5724 leader_election.cc:304] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:45.387989  5797 ts_tablet_manager.cc:1403] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.087s	user 0.028s	sys 0.009s
I20260709 13:14:45.393386  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bd46f0367d4b96ab704bb1a32ab93d" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.393342  5797 raft_consensus.cc:359] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.394263  5797 raft_consensus.cc:385] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.394349  5807 raft_consensus.cc:2804] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.394640  5797 raft_consensus.cc:740] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.395195  5807 raft_consensus.cc:697] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:45.395562  5797 consensus_queue.cc:260] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.396831  5684 raft_consensus.cc:2468] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:45.402657  5797 ts_tablet_manager.cc:1434] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:14:45.403838  5797 tablet_bootstrap.cc:492] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:45.396469  5807 consensus_queue.cc:237] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.415235  5822 raft_consensus.cc:493] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.416246  5822 raft_consensus.cc:515] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.421737  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:45.422459  5609 raft_consensus.cc:2468] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:45.433478  5721 leader_election.cc:304] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.436187  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
W20260709 13:14:45.438143  5724 leader_election.cc:343] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.438831  5822 leader_election.cc:290] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.439486  5822 raft_consensus.cc:2804] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.439900  5822 raft_consensus.cc:493] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.440349  5822 raft_consensus.cc:3060] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.445693  5821 raft_consensus.cc:515] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.449232  5821 leader_election.cc:290] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.449924  5757 raft_consensus.cc:2468] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 1.
I20260709 13:14:45.454366  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bd46f0367d4b96ab704bb1a32ab93d" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:45.455133  5684 raft_consensus.cc:3060] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.455873  5647 leader_election.cc:304] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:45.457270  5803 raft_consensus.cc:2804] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.458271  5799 tablet_bootstrap.cc:492] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:45.460942  5799 ts_tablet_manager.cc:1403] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.100s	user 0.025s	sys 0.016s
I20260709 13:14:45.463671  5684 raft_consensus.cc:2468] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:45.464259  5799 raft_consensus.cc:359] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.460337  5797 tablet_bootstrap.cc:654] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.467114  5506 catalog_manager.cc:5697] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.455976  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59bd46f0367d4b96ab704bb1a32ab93d" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:45.465098  5799 raft_consensus.cc:385] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.465098  5724 leader_election.cc:304] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.468355  5609 raft_consensus.cc:3060] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.468982  5799 raft_consensus.cc:740] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.470436  5798 tablet_bootstrap.cc:492] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:45.469825  5842 raft_consensus.cc:2804] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.471190  5798 ts_tablet_manager.cc:1403] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.175s	user 0.017s	sys 0.030s
I20260709 13:14:45.471254  5842 raft_consensus.cc:697] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:45.472411  5842 consensus_queue.cc:237] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.474571  5799 consensus_queue.cc:260] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.474858  5798 raft_consensus.cc:359] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.475660  5798 raft_consensus.cc:385] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.464615  5822 raft_consensus.cc:515] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.475003  5505 catalog_manager.cc:5697] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 0 to 1, leader changed from <none> to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 1 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.478526  5608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:45.482179  5608 raft_consensus.cc:3060] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.477065  5799 ts_tablet_manager.cc:1434] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:14:45.488109  5803 raft_consensus.cc:697] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:45.489398  5799 tablet_bootstrap.cc:492] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:45.489296  5803 consensus_queue.cc:237] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.490535  5608 raft_consensus.cc:2468] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:45.476016  5798 raft_consensus.cc:740] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.492488  5798 consensus_queue.cc:260] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.494675  5798 ts_tablet_manager.cc:1434] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 13:14:45.495205  5721 leader_election.cc:304] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.495776  5798 tablet_bootstrap.cc:492] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:45.479212  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:45.477872  5822 leader_election.cc:290] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.498939  5822 raft_consensus.cc:2804] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.499462  5822 raft_consensus.cc:697] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:45.479686  5609 raft_consensus.cc:2468] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:45.501127  5797 tablet_bootstrap.cc:492] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:45.495977  5799 tablet_bootstrap.cc:654] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.501832  5797 ts_tablet_manager.cc:1403] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.006s
W20260709 13:14:45.500180  5724 leader_election.cc:343] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.504762  5505 catalog_manager.cc:5697] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.505901  5797 raft_consensus.cc:359] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.506884  5797 raft_consensus.cc:385] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.507220  5797 raft_consensus.cc:740] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.508353  5798 tablet_bootstrap.cc:654] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.508276  5797 consensus_queue.cc:260] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.510969  5797 ts_tablet_manager.cc:1434] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:14:45.511994  5797 tablet_bootstrap.cc:492] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:45.501120  5822 consensus_queue.cc:237] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.518023  5797 tablet_bootstrap.cc:654] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.518236  5799 tablet_bootstrap.cc:492] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:45.518790  5799 ts_tablet_manager.cc:1403] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.003s
I20260709 13:14:45.521907  5799 raft_consensus.cc:359] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.522644  5799 raft_consensus.cc:385] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.522939  5799 raft_consensus.cc:740] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.522939  5506 catalog_manager.cc:5697] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 1 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.529541  5797 tablet_bootstrap.cc:492] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:45.530036  5798 tablet_bootstrap.cc:492] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:45.530187  5797 ts_tablet_manager.cc:1403] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.007s
I20260709 13:14:45.530530  5798 ts_tablet_manager.cc:1403] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.005s
I20260709 13:14:45.533648  5798 raft_consensus.cc:359] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.534353  5798 raft_consensus.cc:385] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.529847  5799 consensus_queue.cc:260] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.534579  5797 raft_consensus.cc:359] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.535473  5797 raft_consensus.cc:385] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.535805  5797 raft_consensus.cc:740] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.536571  5797 consensus_queue.cc:260] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.534653  5798 raft_consensus.cc:740] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.536573  5799 ts_tablet_manager.cc:1434] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260709 13:14:45.538480  5798 consensus_queue.cc:260] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.539054  5797 ts_tablet_manager.cc:1434] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:14:45.540337  5505 catalog_manager.cc:5697] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.542253  5798 ts_tablet_manager.cc:1434] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:14:45.543243  5798 tablet_bootstrap.cc:492] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:45.550119  5798 tablet_bootstrap.cc:654] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.558630  5798 tablet_bootstrap.cc:492] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:45.559182  5798 ts_tablet_manager.cc:1403] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 13:14:45.561595  5798 raft_consensus.cc:359] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.562140  5798 raft_consensus.cc:385] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.562384  5798 raft_consensus.cc:740] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.562999  5798 consensus_queue.cc:260] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.565050  5798 ts_tablet_manager.cc:1434] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:45.606436  5803 raft_consensus.cc:493] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.607029  5803 raft_consensus.cc:515] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.608934  5803 leader_election.cc:290] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.609697  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:45.609889  5757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:45.610445  5609 raft_consensus.cc:2393] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 204d6918cdaf414f9860e5cbc718b0aa in current term 1: Already voted for candidate 61b1acaac8bf494cb41df91d5c6affca in this term.
I20260709 13:14:45.611456  5647 leader_election.cc:304] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa; no voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:45.612160  5803 raft_consensus.cc:3060] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.616571  5822 raft_consensus.cc:493] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.617050  5822 raft_consensus.cc:515] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.617201  5803 raft_consensus.cc:2749] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.618747  5822 leader_election.cc:290] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.619472  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1233b88a15c64dea8fea255c68f3b264" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:45.619658  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1233b88a15c64dea8fea255c68f3b264" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.620045  5609 raft_consensus.cc:2468] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:45.620273  5684 raft_consensus.cc:2468] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:45.621066  5721 leader_election.cc:304] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.621681  5822 raft_consensus.cc:2804] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.622059  5822 raft_consensus.cc:493] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.622326  5822 raft_consensus.cc:3060] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.627040  5822 raft_consensus.cc:515] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.628669  5822 leader_election.cc:290] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:45.629420  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1233b88a15c64dea8fea255c68f3b264" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:45.629739  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1233b88a15c64dea8fea255c68f3b264" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:45.629984  5609 raft_consensus.cc:3060] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.630385  5684 raft_consensus.cc:3060] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.633899  5822 raft_consensus.cc:493] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.634407  5822 raft_consensus.cc:515] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.635689  5609 raft_consensus.cc:2468] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:45.636016  5684 raft_consensus.cc:2468] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:45.636686  5822 leader_election.cc:290] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869)
I20260709 13:14:45.637326  5721 leader_election.cc:304] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.637836  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:45.637894  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.638197  5822 raft_consensus.cc:2804] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.638913  5822 raft_consensus.cc:697] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:45.638978  5684 raft_consensus.cc:2393] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61b1acaac8bf494cb41df91d5c6affca in current term 1: Already voted for candidate 03fb445d9b604085af1649bdedf5f55e in this term.
I20260709 13:14:45.639806  5822 consensus_queue.cc:237] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.640538  5724 leader_election.cc:304] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:45.641325  5842 raft_consensus.cc:3060] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.647898  5842 raft_consensus.cc:2749] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.650274  5506 catalog_manager.cc:5697] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.688244  5802 raft_consensus.cc:493] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.688791  5802 raft_consensus.cc:515] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.690464  5802 leader_election.cc:290] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.691543  5757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:45.691589  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.692184  5757 raft_consensus.cc:2468] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 0.
I20260709 13:14:45.692431  5684 raft_consensus.cc:2468] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 0.
I20260709 13:14:45.693253  5572 leader_election.cc:304] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:45.694162  5828 raft_consensus.cc:2804] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.693782  5802 raft_consensus.cc:493] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.694545  5828 raft_consensus.cc:493] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.695053  5828 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.695012  5802 raft_consensus.cc:515] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.697264  5802 leader_election.cc:290] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.697585  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ba4865f65a49a093a45bacdee5f8cd" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.697855  5757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ba4865f65a49a093a45bacdee5f8cd" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:45.698596  5757 raft_consensus.cc:2393] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 03fb445d9b604085af1649bdedf5f55e in current term 1: Already voted for candidate 204d6918cdaf414f9860e5cbc718b0aa in this term.
I20260709 13:14:45.699524  5572 leader_election.cc:304] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e; no voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:45.700237  5802 raft_consensus.cc:3060] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.701987  5828 raft_consensus.cc:515] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.704548  5828 leader_election.cc:290] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.704813  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:45.705117  5757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:45.705449  5684 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.705678  5757 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.706833  5802 raft_consensus.cc:2749] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.712422  5684 raft_consensus.cc:2468] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 1.
I20260709 13:14:45.712422  5757 raft_consensus.cc:2468] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 1.
I20260709 13:14:45.714066  5575 leader_election.cc:304] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:45.714835  5802 raft_consensus.cc:2804] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.715306  5802 raft_consensus.cc:697] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:45.716089  5802 consensus_queue.cc:237] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.722872  5506 catalog_manager.cc:5697] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 0 to 1, leader changed from <none> to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 1 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.727406  5802 raft_consensus.cc:493] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.727943  5802 raft_consensus.cc:515] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:45.731266  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:45.732031  5683 raft_consensus.cc:2393] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 03fb445d9b604085af1649bdedf5f55e in current term 1: Already voted for candidate 61b1acaac8bf494cb41df91d5c6affca in this term.
I20260709 13:14:45.731917  5802 leader_election.cc:290] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:45.732302  5828 consensus_queue.cc:1048] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.732951  5757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:45.734247  5572 leader_election.cc:304] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e; no voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:45.734958  5828 raft_consensus.cc:3060] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.743183  5828 raft_consensus.cc:2749] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.749675  5802 consensus_queue.cc:1048] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.827281  5842 consensus_queue.cc:1048] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.838009  5842 consensus_queue.cc:1048] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.847806  5842 consensus_queue.cc:1048] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.859949  5842 consensus_queue.cc:1048] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.880486  5802 consensus_queue.cc:1048] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.886632  5758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "03fb445d9b604085af1649bdedf5f55e"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:45.887246  5758 raft_consensus.cc:606] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:45.894508  5802 consensus_queue.cc:1048] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.894577  5803 consensus_queue.cc:1048] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:45.894629  5757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "924c6b951ad34e9f9cc3610764e2f7c3"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "03fb445d9b604085af1649bdedf5f55e"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:45.895485  5757 raft_consensus.cc:606] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:45.918473  5844 consensus_queue.cc:1048] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.922215  5608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde"
dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
mode: GRACEFUL
new_leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:41440
I20260709 13:14:45.922981  5608 raft_consensus.cc:606] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Received request to transfer leadership to TS 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:45.932238  5756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1233b88a15c64dea8fea255c68f3b264"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:45.932739  5756 raft_consensus.cc:606] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:45.938544  5756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:45.939100  5756 raft_consensus.cc:606] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:45.954317  5842 consensus_queue.cc:1048] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.965014  5842 consensus_queue.cc:1048] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.052286  5842 consensus_queue.cc:1048] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.062865  5821 consensus_queue.cc:1048] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.071662  5822 raft_consensus.cc:993] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:46.072183  5821 raft_consensus.cc:1081] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:46.073680  5683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:42068
I20260709 13:14:46.074230  5683 raft_consensus.cc:493] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:46.074510  5683 raft_consensus.cc:3060] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.079385  5683 raft_consensus.cc:515] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.080924  5683 leader_election.cc:290] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:46.081841  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:46.082224  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfad76b6e004e95bc768e524eaa3b62" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:46.082566  5609 raft_consensus.cc:3060] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.082883  5756 raft_consensus.cc:3055] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:46.083220  5756 raft_consensus.cc:740] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:46.084059  5756 consensus_queue.cc:260] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.085310  5756 raft_consensus.cc:3060] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.087941  5609 raft_consensus.cc:2468] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:46.088939  5647 leader_election.cc:304] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:46.089622  5844 raft_consensus.cc:2804] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:46.090006  5844 raft_consensus.cc:697] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:46.090826  5756 raft_consensus.cc:2468] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:46.090728  5844 consensus_queue.cc:237] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.095808  5821 consensus_queue.cc:1048] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.096824  5505 catalog_manager.cc:5697] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 2 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.108266  5807 consensus_queue.cc:1048] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.116412  5842 raft_consensus.cc:993] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:46.116861  5807 raft_consensus.cc:1081] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:46.118366  5683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1233b88a15c64dea8fea255c68f3b264"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:42068
I20260709 13:14:46.118916  5683 raft_consensus.cc:493] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:46.119277  5683 raft_consensus.cc:3060] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.124161  5683 raft_consensus.cc:515] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.126324  5683 leader_election.cc:290] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:46.127050  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1233b88a15c64dea8fea255c68f3b264" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:46.127462  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1233b88a15c64dea8fea255c68f3b264" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:46.127714  5609 raft_consensus.cc:3060] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.128118  5756 raft_consensus.cc:3055] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:46.128443  5756 raft_consensus.cc:740] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:46.129077  5756 consensus_queue.cc:260] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.130020  5756 raft_consensus.cc:3060] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.133026  5609 raft_consensus.cc:2468] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:46.134214  5647 leader_election.cc:304] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:46.135013  5844 raft_consensus.cc:2804] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:46.135481  5844 raft_consensus.cc:697] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:46.135950  5756 raft_consensus.cc:2468] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:46.136348  5844 consensus_queue.cc:237] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.142696  5505 catalog_manager.cc:5697] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 2 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.157086  5802 consensus_queue.cc:1048] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.168599  5802 consensus_queue.cc:1048] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.175557  5828 raft_consensus.cc:993] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e: : Instructing follower 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:46.176023  5802 raft_consensus.cc:1081] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Signalling peer 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:46.177457  5683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:42056
I20260709 13:14:46.178026  5683 raft_consensus.cc:493] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:46.178373  5683 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.183024  5683 raft_consensus.cc:515] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.184828  5683 leader_election.cc:290] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:46.185786  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:46.186197  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:46.186457  5609 raft_consensus.cc:3055] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:46.186802  5756 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.186805  5609 raft_consensus.cc:740] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:46.187865  5609 consensus_queue.cc:260] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.189190  5609 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.193913  5756 raft_consensus.cc:2468] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:46.195161  5647 leader_election.cc:304] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:46.195820  5844 raft_consensus.cc:2804] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:46.195784  5609 raft_consensus.cc:2468] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:46.196597  5844 raft_consensus.cc:697] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:46.197466  5844 consensus_queue.cc:237] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.202925  5505 catalog_manager.cc:5697] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 1 to 2, leader changed from 03fb445d9b604085af1649bdedf5f55e (127.27.63.65) to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 2 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.337113  5842 raft_consensus.cc:993] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 03fb445d9b604085af1649bdedf5f55e to start an election
I20260709 13:14:46.337566  5807 raft_consensus.cc:1081] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 03fb445d9b604085af1649bdedf5f55e to start an election
I20260709 13:14:46.339021  5609 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "924c6b951ad34e9f9cc3610764e2f7c3"
dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
 from {username='slave'} at 127.0.0.1:41434
I20260709 13:14:46.339502  5609 raft_consensus.cc:493] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:46.339811  5609 raft_consensus.cc:3060] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.344973  5609 raft_consensus.cc:515] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.346727  5609 leader_election.cc:290] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 2 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:46.347522  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924c6b951ad34e9f9cc3610764e2f7c3" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:46.347945  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924c6b951ad34e9f9cc3610764e2f7c3" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:46.348228  5683 raft_consensus.cc:3060] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.348573  5756 raft_consensus.cc:3055] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:46.348878  5756 raft_consensus.cc:740] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:46.349665  5756 consensus_queue.cc:260] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.350622  5756 raft_consensus.cc:3060] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.353745  5683 raft_consensus.cc:2468] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 2.
I20260709 13:14:46.354879  5575 leader_election.cc:304] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:46.355614  5802 raft_consensus.cc:2804] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:46.355975  5756 raft_consensus.cc:2468] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 2.
I20260709 13:14:46.356165  5802 raft_consensus.cc:697] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:46.357069  5802 consensus_queue.cc:237] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.363852  5505 catalog_manager.cc:5697] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 2 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.426123  5822 raft_consensus.cc:993] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 03fb445d9b604085af1649bdedf5f55e to start an election
I20260709 13:14:46.426642  5807 raft_consensus.cc:1081] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 03fb445d9b604085af1649bdedf5f55e to start an election
I20260709 13:14:46.428225  5609 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f"
dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
 from {username='slave'} at 127.0.0.1:41434
I20260709 13:14:46.428730  5609 raft_consensus.cc:493] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:46.429042  5609 raft_consensus.cc:3060] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.434062  5609 raft_consensus.cc:515] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.435631  5609 leader_election.cc:290] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 2 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:46.436510  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:46.436710  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f59faadac8a40caad4aead074d3ad1f" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:46.437104  5683 raft_consensus.cc:3060] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.437284  5756 raft_consensus.cc:3055] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:46.437662  5756 raft_consensus.cc:740] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:46.438355  5756 consensus_queue.cc:260] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.439280  5756 raft_consensus.cc:3060] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.442096  5683 raft_consensus.cc:2468] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 2.
I20260709 13:14:46.443109  5575 leader_election.cc:304] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:46.443886  5802 raft_consensus.cc:2804] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:46.444396  5802 raft_consensus.cc:697] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:46.444427  5756 raft_consensus.cc:2468] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 2.
I20260709 13:14:46.445350  5802 consensus_queue.cc:237] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:46.451930  5503 catalog_manager.cc:5697] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 2 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.546113  5609 raft_consensus.cc:1275] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.547288  5880 consensus_queue.cc:1048] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:46.555706  5756 raft_consensus.cc:1275] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.557255  5880 consensus_queue.cc:1048] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:46.634565  5756 raft_consensus.cc:1275] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.635809  5880 consensus_queue.cc:1048] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:46.643750  5609 raft_consensus.cc:1275] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.645469  5880 consensus_queue.cc:1048] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:46.701289  5609 raft_consensus.cc:1275] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.702571  5803 consensus_queue.cc:1048] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:46.703823  5756 raft_consensus.cc:1275] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.705240  5803 consensus_queue.cc:1048] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:46.866947  5683 raft_consensus.cc:1275] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Refusing update from remote peer 03fb445d9b604085af1649bdedf5f55e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.868379  5832 consensus_queue.cc:1048] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:46.878675  5756 raft_consensus.cc:1275] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 03fb445d9b604085af1649bdedf5f55e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.879217  5758 raft_consensus.cc:1275] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 03fb445d9b604085af1649bdedf5f55e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.880576  5879 consensus_queue.cc:1048] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:46.880614  5802 consensus_queue.cc:1048] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:46.893829  5683 raft_consensus.cc:1275] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Refusing update from remote peer 03fb445d9b604085af1649bdedf5f55e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:46.895602  5879 consensus_queue.cc:1048] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.968951  5505 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47618:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:48.971227  5505 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:49.016579  5594 tablet_service.cc:1511] Processing CreateTablet for tablet 13a4ca4d476d4a7fb07defd2dfcb67a7 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:49.017859  5594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13a4ca4d476d4a7fb07defd2dfcb67a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.017771  5668 tablet_service.cc:1511] Processing CreateTablet for tablet 13a4ca4d476d4a7fb07defd2dfcb67a7 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:49.019140  5668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13a4ca4d476d4a7fb07defd2dfcb67a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.018996  5744 tablet_service.cc:1511] Processing CreateTablet for tablet 13a4ca4d476d4a7fb07defd2dfcb67a7 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:49.020273  5744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13a4ca4d476d4a7fb07defd2dfcb67a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.025492  5670 tablet_service.cc:1511] Processing CreateTablet for tablet 5dce9f9c479c4d7a95cc75044ece77a7 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:49.026885  5670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dce9f9c479c4d7a95cc75044ece77a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.033013  5667 tablet_service.cc:1511] Processing CreateTablet for tablet 8bada308d1e64b90adbc36c78c4d0654 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:49.034400  5667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bada308d1e64b90adbc36c78c4d0654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.039510  5674 tablet_service.cc:1511] Processing CreateTablet for tablet 04df0d979cc246009e861b3bcb62016f (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:49.040529  5672 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3f7439b0f949debb699c92a65c3df5 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:49.040702  5666 tablet_service.cc:1511] Processing CreateTablet for tablet aca642f543974064bbacc6639ad410ac (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:49.042652  5672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3f7439b0f949debb699c92a65c3df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.043372  5666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aca642f543974064bbacc6639ad410ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.045293  5674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04df0d979cc246009e861b3bcb62016f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.046509  5673 tablet_service.cc:1511] Processing CreateTablet for tablet 859cc219194843cbbe50c4e29eb66328 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:49.050555  5598 tablet_service.cc:1511] Processing CreateTablet for tablet 5dce9f9c479c4d7a95cc75044ece77a7 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:49.051952  5598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dce9f9c479c4d7a95cc75044ece77a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.053651  5748 tablet_service.cc:1511] Processing CreateTablet for tablet 5dce9f9c479c4d7a95cc75044ece77a7 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:49.054422  5742 tablet_service.cc:1511] Processing CreateTablet for tablet aca642f543974064bbacc6639ad410ac (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:49.054987  5748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dce9f9c479c4d7a95cc75044ece77a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.055644  5742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aca642f543974064bbacc6639ad410ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.055543  5745 tablet_service.cc:1511] Processing CreateTablet for tablet 8bada308d1e64b90adbc36c78c4d0654 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:49.056875  5740 tablet_service.cc:1511] Processing CreateTablet for tablet 04df0d979cc246009e861b3bcb62016f (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:49.061429  5595 tablet_service.cc:1511] Processing CreateTablet for tablet 8bada308d1e64b90adbc36c78c4d0654 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:49.061722  5593 tablet_service.cc:1511] Processing CreateTablet for tablet aca642f543974064bbacc6639ad410ac (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:49.062673  5595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bada308d1e64b90adbc36c78c4d0654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.063688  5596 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3f7439b0f949debb699c92a65c3df5 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:49.064337  5740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04df0d979cc246009e861b3bcb62016f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.064963  5596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3f7439b0f949debb699c92a65c3df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.069039  5597 tablet_service.cc:1511] Processing CreateTablet for tablet 04df0d979cc246009e861b3bcb62016f (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:49.070322  5597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04df0d979cc246009e861b3bcb62016f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.074620  5592 tablet_service.cc:1511] Processing CreateTablet for tablet 859cc219194843cbbe50c4e29eb66328 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:49.075945  5593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aca642f543974064bbacc6639ad410ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.080271  5745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bada308d1e64b90adbc36c78c4d0654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.080312  5592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859cc219194843cbbe50c4e29eb66328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.083957  5673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859cc219194843cbbe50c4e29eb66328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.092947  5741 tablet_service.cc:1511] Processing CreateTablet for tablet 859cc219194843cbbe50c4e29eb66328 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:49.094321  5741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859cc219194843cbbe50c4e29eb66328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.097417  5743 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3f7439b0f949debb699c92a65c3df5 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:49.098696  5743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3f7439b0f949debb699c92a65c3df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.114166  5591 tablet_service.cc:1511] Processing CreateTablet for tablet 05634e54fd08425ea500f4f4a0dc145d (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:49.115578  5591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05634e54fd08425ea500f4f4a0dc145d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.135562  5670 tablet_service.cc:1511] Processing CreateTablet for tablet 05634e54fd08425ea500f4f4a0dc145d (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:49.136974  5670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05634e54fd08425ea500f4f4a0dc145d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.139788  5897 tablet_bootstrap.cc:492] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.139906  5669 tablet_service.cc:1511] Processing CreateTablet for tablet 2f9eb71b83dd4911ac125e54e2ce8466 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:49.141278  5669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f9eb71b83dd4911ac125e54e2ce8466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.154102  5896 tablet_bootstrap.cc:492] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.158262  5596 tablet_service.cc:1511] Processing CreateTablet for tablet 2f9eb71b83dd4911ac125e54e2ce8466 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:49.159653  5596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f9eb71b83dd4911ac125e54e2ce8466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.163422  5746 tablet_service.cc:1511] Processing CreateTablet for tablet 2f9eb71b83dd4911ac125e54e2ce8466 (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:49.164750  5746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f9eb71b83dd4911ac125e54e2ce8466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.168581  5897 tablet_bootstrap.cc:654] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.181351  5748 tablet_service.cc:1511] Processing CreateTablet for tablet 05634e54fd08425ea500f4f4a0dc145d (DEFAULT_TABLE table=soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:49.185607  5896 tablet_bootstrap.cc:654] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.195122  5899 tablet_bootstrap.cc:492] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.197306  5748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05634e54fd08425ea500f4f4a0dc145d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:49.197542  5897 tablet_bootstrap.cc:492] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.199548  5897 ts_tablet_manager.cc:1403] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.002s
I20260709 13:14:49.202636  5897 raft_consensus.cc:359] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.203397  5897 raft_consensus.cc:385] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.203756  5897 raft_consensus.cc:740] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.204867  5897 consensus_queue.cc:260] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.206475  5899 tablet_bootstrap.cc:654] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.207027  5897 ts_tablet_manager.cc:1434] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:14:49.208078  5897 tablet_bootstrap.cc:492] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.215320  5897 tablet_bootstrap.cc:654] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.218708  5896 tablet_bootstrap.cc:492] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.219200  5896 ts_tablet_manager.cc:1403] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.065s	user 0.003s	sys 0.013s
I20260709 13:14:49.220820  5899 tablet_bootstrap.cc:492] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.221319  5899 ts_tablet_manager.cc:1403] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.012s
I20260709 13:14:49.222154  5896 raft_consensus.cc:359] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.223033  5896 raft_consensus.cc:385] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.223415  5896 raft_consensus.cc:740] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.224754  5896 consensus_queue.cc:260] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.225641  5879 raft_consensus.cc:493] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.226084  5879 raft_consensus.cc:515] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.226959  5897 tablet_bootstrap.cc:492] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.227506  5897 ts_tablet_manager.cc:1403] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.007s
I20260709 13:14:49.227815  5899 raft_consensus.cc:359] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.228605  5899 raft_consensus.cc:385] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.228874  5899 raft_consensus.cc:740] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.229669  5899 consensus_queue.cc:260] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.230599  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13a4ca4d476d4a7fb07defd2dfcb67a7" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.230499  5897 raft_consensus.cc:359] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.231314  5897 raft_consensus.cc:385] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.231611  5897 raft_consensus.cc:740] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.231873  5899 ts_tablet_manager.cc:1434] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:14:49.232345  5897 consensus_queue.cc:260] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.233187  5899 tablet_bootstrap.cc:492] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.234230  5896 ts_tablet_manager.cc:1434] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:14:49.234613  5897 ts_tablet_manager.cc:1434] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:14:49.235216  5896 tablet_bootstrap.cc:492] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.235567  5897 tablet_bootstrap.cc:492] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.231323  5683 raft_consensus.cc:2468] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 0.
I20260709 13:14:49.238303  5575 leader_election.cc:304] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:49.239153  5898 raft_consensus.cc:2804] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.239488  5898 raft_consensus.cc:493] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.239809  5899 tablet_bootstrap.cc:654] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.242429  5896 tablet_bootstrap.cc:654] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.239825  5898 raft_consensus.cc:3060] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.242424  5897 tablet_bootstrap.cc:654] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.245312  5899 tablet_bootstrap.cc:492] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.245793  5899 ts_tablet_manager.cc:1403] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.001s
I20260709 13:14:49.248572  5899 raft_consensus.cc:359] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.248802  5879 leader_election.cc:290] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.249307  5899 raft_consensus.cc:385] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.249743  5899 raft_consensus.cc:740] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.250365  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13a4ca4d476d4a7fb07defd2dfcb67a7" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:49.250464  5899 consensus_queue.cc:260] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
W20260709 13:14:49.252393  5572 leader_election.cc:343] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.252825  5899 ts_tablet_manager.cc:1434] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:49.253036  5898 raft_consensus.cc:515] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.253856  5899 tablet_bootstrap.cc:492] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.255829  5898 leader_election.cc:290] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.257627  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13a4ca4d476d4a7fb07defd2dfcb67a7" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:49.258234  5683 raft_consensus.cc:3060] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.259166  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13a4ca4d476d4a7fb07defd2dfcb67a7" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
W20260709 13:14:49.260608  5572 leader_election.cc:343] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.263104  5897 tablet_bootstrap.cc:492] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.263607  5897 ts_tablet_manager.cc:1403] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.006s
I20260709 13:14:49.263772  5899 tablet_bootstrap.cc:654] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.265729  5683 raft_consensus.cc:2468] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 1.
I20260709 13:14:49.266927  5575 leader_election.cc:304] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:49.268178  5898 raft_consensus.cc:2804] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.268633  5898 raft_consensus.cc:697] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:49.269407  5898 consensus_queue.cc:237] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.267177  5897 raft_consensus.cc:359] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.274279  5897 raft_consensus.cc:385] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.274556  5897 raft_consensus.cc:740] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.275202  5897 consensus_queue.cc:260] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.277499  5897 ts_tablet_manager.cc:1434] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 13:14:49.278476  5897 tablet_bootstrap.cc:492] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.288888  5897 tablet_bootstrap.cc:654] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:49.290256  5555 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:49.290562  5899 tablet_bootstrap.cc:492] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.291081  5899 ts_tablet_manager.cc:1403] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.017s
I20260709 13:14:49.294203  5899 raft_consensus.cc:359] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.294927  5899 raft_consensus.cc:385] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.295215  5899 raft_consensus.cc:740] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.295768  5896 tablet_bootstrap.cc:492] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.295918  5899 consensus_queue.cc:260] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.296586  5896 ts_tablet_manager.cc:1403] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.062s	user 0.016s	sys 0.028s
I20260709 13:14:49.298390  5899 ts_tablet_manager.cc:1434] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:49.299345  5899 tablet_bootstrap.cc:492] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.304781  5896 raft_consensus.cc:359] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.305848  5896 raft_consensus.cc:385] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.306339  5899 tablet_bootstrap.cc:654] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.306217  5896 raft_consensus.cc:740] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.309933  5896 consensus_queue.cc:260] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.310178  5507 catalog_manager.cc:5697] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 0 to 1, leader changed from <none> to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 1 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:49.313519  5896 ts_tablet_manager.cc:1434] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 13:14:49.317832  5896 tablet_bootstrap.cc:492] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.331303  5896 tablet_bootstrap.cc:654] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.340219  5897 tablet_bootstrap.cc:492] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.340644  5896 tablet_bootstrap.cc:492] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.340744  5897 ts_tablet_manager.cc:1403] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.062s	user 0.029s	sys 0.024s
I20260709 13:14:49.340934  5899 tablet_bootstrap.cc:492] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.341303  5896 ts_tablet_manager.cc:1403] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.003s
I20260709 13:14:49.341488  5899 ts_tablet_manager.cc:1403] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.018s
I20260709 13:14:49.343864  5899 raft_consensus.cc:359] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.344584  5899 raft_consensus.cc:385] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.344455  5897 raft_consensus.cc:359] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.345074  5899 raft_consensus.cc:740] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.344761  5896 raft_consensus.cc:359] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.345266  5897 raft_consensus.cc:385] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.345623  5896 raft_consensus.cc:385] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.345645  5897 raft_consensus.cc:740] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.345973  5896 raft_consensus.cc:740] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.345938  5899 consensus_queue.cc:260] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.347007  5897 consensus_queue.cc:260] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.348865  5899 ts_tablet_manager.cc:1434] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:49.349565  5897 ts_tablet_manager.cc:1434] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:14:49.350591  5897 tablet_bootstrap.cc:492] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.347318  5896 consensus_queue.cc:260] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.351119  5899 tablet_bootstrap.cc:492] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.352845  5896 ts_tablet_manager.cc:1434] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:14:49.354214  5896 tablet_bootstrap.cc:492] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.354377  5822 raft_consensus.cc:493] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.354933  5822 raft_consensus.cc:515] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.357151  5822 leader_election.cc:290] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.357211  5897 tablet_bootstrap.cc:654] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.358093  5899 tablet_bootstrap.cc:654] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.358731  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca642f543974064bbacc6639ad410ac" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
W20260709 13:14:49.360417  5721 leader_election.cc:343] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.361217  5896 tablet_bootstrap.cc:654] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.362545  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca642f543974064bbacc6639ad410ac" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.363435  5683 raft_consensus.cc:2468] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:49.366860  5724 leader_election.cc:304] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:49.368269  5822 raft_consensus.cc:2804] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.368629  5822 raft_consensus.cc:493] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.368939  5822 raft_consensus.cc:3060] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.370863  5897 tablet_bootstrap.cc:492] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.371574  5897 ts_tablet_manager.cc:1403] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.008s
I20260709 13:14:49.378245  5899 tablet_bootstrap.cc:492] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.378979  5899 ts_tablet_manager.cc:1403] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.008s
I20260709 13:14:49.382539  5822 raft_consensus.cc:515] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.382966  5899 raft_consensus.cc:359] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.383963  5899 raft_consensus.cc:385] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.384361  5899 raft_consensus.cc:740] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.385342  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca642f543974064bbacc6639ad410ac" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:49.385315  5899 consensus_queue.cc:260] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
W20260709 13:14:49.386783  5721 leader_election.cc:343] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.375500  5897 raft_consensus.cc:359] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.387761  5897 raft_consensus.cc:385] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.388185  5897 raft_consensus.cc:740] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.388798  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca642f543974064bbacc6639ad410ac" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:49.389410  5683 raft_consensus.cc:3060] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.389137  5897 consensus_queue.cc:260] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.391647  5897 ts_tablet_manager.cc:1434] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:14:49.392616  5896 tablet_bootstrap.cc:492] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.393293  5896 ts_tablet_manager.cc:1403] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.018s
I20260709 13:14:49.396859  5683 raft_consensus.cc:2468] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:49.397231  5896 raft_consensus.cc:359] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.397977  5724 leader_election.cc:304] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:49.398241  5896 raft_consensus.cc:385] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.398695  5896 raft_consensus.cc:740] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.399613  5896 consensus_queue.cc:260] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.402624  5896 ts_tablet_manager.cc:1434] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:14:49.403910  5896 tablet_bootstrap.cc:492] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.405804  5822 leader_election.cc:290] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.406661  5822 raft_consensus.cc:2804] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.407150  5822 raft_consensus.cc:697] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:49.392658  5897 tablet_bootstrap.cc:492] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.414584  5896 tablet_bootstrap.cc:654] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.407970  5822 consensus_queue.cc:237] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.421872  5897 tablet_bootstrap.cc:654] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.449749  5899 ts_tablet_manager.cc:1434] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.070s	user 0.028s	sys 0.038s
I20260709 13:14:49.451324  5899 tablet_bootstrap.cc:492] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.453783  5896 tablet_bootstrap.cc:492] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.454382  5897 tablet_bootstrap.cc:492] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.454561  5896 ts_tablet_manager.cc:1403] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.037s
I20260709 13:14:49.454756  5898 raft_consensus.cc:493] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.455116  5897 ts_tablet_manager.cc:1403] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.063s	user 0.007s	sys 0.011s
I20260709 13:14:49.455444  5898 raft_consensus.cc:515] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.458047  5899 tablet_bootstrap.cc:654] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.458357  5897 raft_consensus.cc:359] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.459141  5897 raft_consensus.cc:385] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.458696  5896 raft_consensus.cc:359] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.459460  5897 raft_consensus.cc:740] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.459702  5896 raft_consensus.cc:385] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.460302  5896 raft_consensus.cc:740] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.460525  5898 leader_election.cc:290] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.460273  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dce9f9c479c4d7a95cc75044ece77a7" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.460273  5897 consensus_queue.cc:260] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.461778  5683 raft_consensus.cc:2468] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 0.
I20260709 13:14:49.462615  5896 consensus_queue.cc:260] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.468315  5896 ts_tablet_manager.cc:1434] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:14:49.463725  5575 leader_election.cc:304] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:49.459781  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dce9f9c479c4d7a95cc75044ece77a7" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:49.465591  5897 ts_tablet_manager.cc:1434] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:14:49.469523  5896 tablet_bootstrap.cc:492] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.469976  5756 raft_consensus.cc:2468] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 0.
I20260709 13:14:49.470319  5899 tablet_bootstrap.cc:492] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.469547  5898 raft_consensus.cc:2804] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.471014  5898 raft_consensus.cc:493] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.471076  5899 ts_tablet_manager.cc:1403] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:14:49.471442  5898 raft_consensus.cc:3060] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.471894  5897 tablet_bootstrap.cc:492] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.474462  5899 raft_consensus.cc:359] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.475191  5899 raft_consensus.cc:385] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.475502  5899 raft_consensus.cc:740] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.475701  5503 catalog_manager.cc:5697] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.476373  5899 consensus_queue.cc:260] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.477036  5898 raft_consensus.cc:515] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.478464  5897 tablet_bootstrap.cc:654] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.480232  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dce9f9c479c4d7a95cc75044ece77a7" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:49.480773  5683 raft_consensus.cc:3060] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.487206  5899 ts_tablet_manager.cc:1434] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:14:49.487682  5896 tablet_bootstrap.cc:654] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.488363  5899 tablet_bootstrap.cc:492] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.496428  5880 raft_consensus.cc:493] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.496667  5899 tablet_bootstrap.cc:654] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.497007  5880 raft_consensus.cc:515] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.499917  5880 leader_election.cc:290] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.501700  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04df0d979cc246009e861b3bcb62016f" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:49.502359  5609 raft_consensus.cc:2468] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 0.
I20260709 13:14:49.503329  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04df0d979cc246009e861b3bcb62016f" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:49.505590  5683 raft_consensus.cc:2468] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 1.
I20260709 13:14:49.506795  5575 leader_election.cc:304] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:49.507648  5879 raft_consensus.cc:2804] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.508741  5647 leader_election.cc:304] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
W20260709 13:14:49.509977  5647 leader_election.cc:343] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.513616  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dce9f9c479c4d7a95cc75044ece77a7" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:49.514250  5756 raft_consensus.cc:3060] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.515998  5822 raft_consensus.cc:493] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.516557  5822 raft_consensus.cc:515] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.518810  5822 leader_election.cc:290] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.520783  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.521526  5684 raft_consensus.cc:2468] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:49.523149  5724 leader_election.cc:304] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:49.524612  5822 raft_consensus.cc:2804] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.525077  5822 raft_consensus.cc:493] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.525620  5915 raft_consensus.cc:493] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.525838  5822 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.526176  5915 raft_consensus.cc:515] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.528406  5915 leader_election.cc:290] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.530105  5880 raft_consensus.cc:2804] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.530009  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.534687  5880 raft_consensus.cc:493] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.533316  5916 raft_consensus.cc:493] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.530691  5915 raft_consensus.cc:493] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.536831  5915 raft_consensus.cc:515] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
W20260709 13:14:49.535730  5724 leader_election.cc:343] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.536799  5916 raft_consensus.cc:515] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.539285  5915 leader_election.cc:290] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.540596  5916 leader_election.cc:290] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.542647  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13a4ca4d476d4a7fb07defd2dfcb67a7" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.543064  5880 raft_consensus.cc:3060] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.543561  5684 raft_consensus.cc:2393] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61b1acaac8bf494cb41df91d5c6affca in current term 1: Already voted for candidate 03fb445d9b604085af1649bdedf5f55e in this term.
I20260709 13:14:49.543175  5682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.544582  5682 raft_consensus.cc:2468] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:49.545820  5724 leader_election.cc:304] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:49.547307  5898 leader_election.cc:290] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.547642  5916 raft_consensus.cc:2804] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.548403  5916 raft_consensus.cc:493] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.548796  5916 raft_consensus.cc:3060] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.556437  5756 raft_consensus.cc:2468] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 1.
I20260709 13:14:49.579449  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:49.581471  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:49.582376  5609 raft_consensus.cc:2468] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:49.582722  5897 tablet_bootstrap.cc:492] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.583583  5897 ts_tablet_manager.cc:1403] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.112s	user 0.027s	sys 0.027s
I20260709 13:14:49.584462  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13a4ca4d476d4a7fb07defd2dfcb67a7" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
W20260709 13:14:49.586968  5721 leader_election.cc:343] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.587930  5897 raft_consensus.cc:359] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.589074  5897 raft_consensus.cc:385] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.588840  5721 leader_election.cc:304] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:49.589510  5897 raft_consensus.cc:740] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.590011  5915 raft_consensus.cc:2804] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.590384  5915 raft_consensus.cc:493] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.590770  5915 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.590477  5897 consensus_queue.cc:260] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.594624  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:49.598925  5721 leader_election.cc:304] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:49.609270  5879 raft_consensus.cc:697] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:49.610546  5879 consensus_queue.cc:237] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.614128  5915 raft_consensus.cc:515] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.618685  5931 raft_consensus.cc:3060] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.571100  5822 raft_consensus.cc:515] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.627982  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:49.628605  5683 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:49.629369  5721 leader_election.cc:343] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.630956  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:49.631242  5608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:49.631577  5609 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:49.632606  5721 leader_election.cc:343] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.634794  5880 raft_consensus.cc:515] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.635303  5683 raft_consensus.cc:2468] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:49.644026  5724 leader_election.cc:304] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:49.635565  5822 leader_election.cc:290] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.645994  5822 raft_consensus.cc:2804] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.646420  5822 raft_consensus.cc:697] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:49.646277  5609 raft_consensus.cc:2468] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:49.649991  5897 ts_tablet_manager.cc:1434] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.066s	user 0.023s	sys 0.017s
I20260709 13:14:49.650271  5721 leader_election.cc:304] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:49.651517  5897 tablet_bootstrap.cc:492] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.636696  5916 raft_consensus.cc:515] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.638063  5896 tablet_bootstrap.cc:492] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.654731  5896 ts_tablet_manager.cc:1403] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.185s	user 0.041s	sys 0.037s
I20260709 13:14:49.656831  5916 leader_election.cc:290] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.641845  5915 leader_election.cc:290] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.642675  5931 raft_consensus.cc:2749] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:49.657932  5896 raft_consensus.cc:359] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.669169  5896 raft_consensus.cc:385] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.669481  5896 raft_consensus.cc:740] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.669586  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:49.641371  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:49.670181  5683 raft_consensus.cc:3060] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.670285  5896 consensus_queue.cc:260] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
W20260709 13:14:49.671175  5724 leader_election.cc:343] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.672914  5896 ts_tablet_manager.cc:1434] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 13:14:49.673890  5896 tablet_bootstrap.cc:492] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.677242  5683 raft_consensus.cc:2468] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:49.678579  5724 leader_election.cc:304] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:49.679752  5931 raft_consensus.cc:2804] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.680284  5931 raft_consensus.cc:697] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:49.680434  5924 raft_consensus.cc:493] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.680936  5896 tablet_bootstrap.cc:654] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.660394  5897 tablet_bootstrap.cc:654] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.681028  5924 raft_consensus.cc:515] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.653887  5503 catalog_manager.cc:5697] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 0 to 1, leader changed from <none> to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 1 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:49.647361  5822 consensus_queue.cc:237] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.663416  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04df0d979cc246009e861b3bcb62016f" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:49.686942  5609 raft_consensus.cc:3060] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.665843  5880 leader_election.cc:290] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.666780  5899 tablet_bootstrap.cc:492] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.667007  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04df0d979cc246009e861b3bcb62016f" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:49.667557  5934 raft_consensus.cc:2804] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.692642  5899 ts_tablet_manager.cc:1403] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.204s	user 0.054s	sys 0.085s
I20260709 13:14:49.693012  5934 raft_consensus.cc:697] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:49.693904  5934 consensus_queue.cc:237] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.696548  5899 raft_consensus.cc:359] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
W20260709 13:14:49.697526  5647 leader_election.cc:343] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.697743  5899 raft_consensus.cc:385] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.696616  5879 consensus_queue.cc:1048] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:49.698228  5899 raft_consensus.cc:740] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.699298  5899 consensus_queue.cc:260] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.702252  5899 ts_tablet_manager.cc:1434] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:14:49.703647  5899 tablet_bootstrap.cc:492] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.704790  5927 consensus_queue.cc:1048] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:49.713245  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
W20260709 13:14:49.719043  5575 leader_election.cc:343] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.719789  5924 leader_election.cc:290] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.715679  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:49.720249  5899 tablet_bootstrap.cc:654] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:49.721993  5572 leader_election.cc:343] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.722553  5572 leader_election.cc:304] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e; no voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:49.723683  5927 raft_consensus.cc:2749] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:49.681192  5931 consensus_queue.cc:237] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.715390  5609 raft_consensus.cc:2468] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 1.
I20260709 13:14:49.731205  5647 leader_election.cc:304] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:49.731788  5899 tablet_bootstrap.cc:492] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.732489  5899 ts_tablet_manager.cc:1403] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.001s
I20260709 13:14:49.732693  5880 raft_consensus.cc:2804] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.733345  5880 raft_consensus.cc:697] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:49.734323  5880 consensus_queue.cc:237] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.742113  5899 raft_consensus.cc:359] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.743170  5899 raft_consensus.cc:385] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.743511  5899 raft_consensus.cc:740] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.744347  5899 consensus_queue.cc:260] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.752933  5899 ts_tablet_manager.cc:1434] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 13:14:49.756390  5899 tablet_bootstrap.cc:492] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: Bootstrap starting.
I20260709 13:14:49.773213  5899 tablet_bootstrap.cc:654] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.792575  5899 tablet_bootstrap.cc:492] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: No bootstrap required, opened a new log
I20260709 13:14:49.793236  5899 ts_tablet_manager.cc:1403] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.000s
I20260709 13:14:49.802003  5899 raft_consensus.cc:359] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.802915  5899 raft_consensus.cc:385] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.803287  5899 raft_consensus.cc:740] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.804225  5899 consensus_queue.cc:260] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.807029  5899 ts_tablet_manager.cc:1434] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:14:49.807757  5507 catalog_manager.cc:5697] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 1 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:49.818104  5897 tablet_bootstrap.cc:492] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.818665  5897 ts_tablet_manager.cc:1403] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.167s	user 0.052s	sys 0.087s
I20260709 13:14:49.819171  5896 tablet_bootstrap.cc:492] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.819694  5896 ts_tablet_manager.cc:1403] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.146s	user 0.025s	sys 0.046s
W20260709 13:14:49.818117  5721 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.65:46869, user_credentials={real_user=slave}} blocked reactor thread for 167932us
I20260709 13:14:49.821347  5897 raft_consensus.cc:359] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.822050  5897 raft_consensus.cc:385] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.822228  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:49.822301  5897 raft_consensus.cc:740] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.822412  5896 raft_consensus.cc:359] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.823207  5896 raft_consensus.cc:385] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.823633  5896 raft_consensus.cc:740] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.823230  5897 consensus_queue.cc:260] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.824450  5896 consensus_queue.cc:260] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.826238  5897 ts_tablet_manager.cc:1434] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:49.826583  5896 ts_tablet_manager.cc:1434] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:14:49.827203  5897 tablet_bootstrap.cc:492] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa: Bootstrap starting.
I20260709 13:14:49.827525  5896 tablet_bootstrap.cc:492] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.833515  5897 tablet_bootstrap.cc:654] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:49.829857  5721 leader_election.cc:343] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.842600  5896 tablet_bootstrap.cc:654] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.839395  5505 catalog_manager.cc:5697] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.844762  5897 tablet_bootstrap.cc:492] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa: No bootstrap required, opened a new log
I20260709 13:14:49.844403  5505 catalog_manager.cc:5697] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.845338  5897 ts_tablet_manager.cc:1403] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.006s
I20260709 13:14:49.846421  5505 catalog_manager.cc:5697] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.849215  5897 raft_consensus.cc:359] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.850189  5897 raft_consensus.cc:385] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.850543  5897 raft_consensus.cc:740] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.851346  5897 consensus_queue.cc:260] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.853896  5897 ts_tablet_manager.cc:1434] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:14:49.854734  5896 tablet_bootstrap.cc:492] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.855265  5896 ts_tablet_manager.cc:1403] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.028s	user 0.004s	sys 0.012s
I20260709 13:14:49.857488  5896 raft_consensus.cc:359] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.858075  5896 raft_consensus.cc:385] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.858301  5896 raft_consensus.cc:740] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.858886  5896 consensus_queue.cc:260] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.862435  5896 ts_tablet_manager.cc:1434] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:49.863658  5896 tablet_bootstrap.cc:492] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e: Bootstrap starting.
I20260709 13:14:49.870725  5896 tablet_bootstrap.cc:654] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.874670  5896 tablet_bootstrap.cc:492] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e: No bootstrap required, opened a new log
I20260709 13:14:49.875090  5896 ts_tablet_manager.cc:1403] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e: Time spent bootstrapping tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:14:49.877734  5896 raft_consensus.cc:359] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.878378  5896 raft_consensus.cc:385] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.878595  5896 raft_consensus.cc:740] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.879171  5896 consensus_queue.cc:260] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.881110  5896 ts_tablet_manager.cc:1434] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:49.892071  5880 raft_consensus.cc:493] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.892557  5880 raft_consensus.cc:515] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.894300  5880 leader_election.cc:290] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.894882  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f9eb71b83dd4911ac125e54e2ce8466" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:49.895241  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f9eb71b83dd4911ac125e54e2ce8466" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:49.895594  5609 raft_consensus.cc:2468] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 0.
I20260709 13:14:49.895869  5756 raft_consensus.cc:2468] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 0.
I20260709 13:14:49.896708  5647 leader_election.cc:304] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:49.897384  5880 raft_consensus.cc:2804] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.897722  5880 raft_consensus.cc:493] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.898063  5880 raft_consensus.cc:3060] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.897838  5822 raft_consensus.cc:493] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.898554  5822 raft_consensus.cc:515] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.900688  5822 leader_election.cc:290] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.902200  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04df0d979cc246009e861b3bcb62016f" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.903038  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04df0d979cc246009e861b3bcb62016f" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:49.903829  5609 raft_consensus.cc:2393] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61b1acaac8bf494cb41df91d5c6affca in current term 1: Already voted for candidate 204d6918cdaf414f9860e5cbc718b0aa in this term.
I20260709 13:14:49.904783  5880 raft_consensus.cc:515] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.905078  5721 leader_election.cc:304] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:49.906298  5822 raft_consensus.cc:3060] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.907662  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f9eb71b83dd4911ac125e54e2ce8466" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:49.908300  5609 raft_consensus.cc:3060] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.908352  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f9eb71b83dd4911ac125e54e2ce8466" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:49.909014  5756 raft_consensus.cc:3060] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.912269  5935 raft_consensus.cc:493] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.912767  5935 raft_consensus.cc:515] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.913232  5822 raft_consensus.cc:2749] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:49.914923  5935 leader_election.cc:290] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.915820  5880 leader_election.cc:290] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:49.915707  5608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:49.915740  5756 raft_consensus.cc:2468] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 1.
I20260709 13:14:49.916500  5608 raft_consensus.cc:2468] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:49.916779  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:49.917438  5684 raft_consensus.cc:2468] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 0.
I20260709 13:14:49.917469  5647 leader_election.cc:304] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:49.917789  5721 leader_election.cc:304] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:49.918572  5880 raft_consensus.cc:2804] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.919068  5880 raft_consensus.cc:697] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:49.919265  5935 raft_consensus.cc:2804] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.919723  5935 raft_consensus.cc:493] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.920097  5935 raft_consensus.cc:3060] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.919870  5880 consensus_queue.cc:237] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.926467  5822 consensus_queue.cc:1048] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:49.928525  5503 catalog_manager.cc:5697] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 1 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:49.935956  5608 raft_consensus.cc:3060] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.940289  5935 raft_consensus.cc:515] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.942463  5935 leader_election.cc:290] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:49.943526  5609 raft_consensus.cc:2468] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 1.
I20260709 13:14:49.943898  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:49.944532  5683 raft_consensus.cc:3060] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.943722  5607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:49.945245  5607 raft_consensus.cc:3060] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.949218  5915 consensus_queue.cc:1048] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:49.952394  5683 raft_consensus.cc:2468] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:49.953591  5724 leader_election.cc:304] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:49.954453  5915 raft_consensus.cc:2804] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.954886  5915 raft_consensus.cc:697] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:49.955643  5915 consensus_queue.cc:237] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:49.958117  5607 raft_consensus.cc:2468] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 1.
I20260709 13:14:49.974754  5503 catalog_manager.cc:5697] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 1 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:50.008685  5924 raft_consensus.cc:493] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.009114  5924 raft_consensus.cc:515] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.010872  5924 leader_election.cc:290] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:50.011557  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:50.011785  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:50.012439  5683 raft_consensus.cc:2393] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 03fb445d9b604085af1649bdedf5f55e in current term 1: Already voted for candidate 61b1acaac8bf494cb41df91d5c6affca in this term.
I20260709 13:14:50.012853  5924 raft_consensus.cc:493] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.013398  5924 raft_consensus.cc:515] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.013695  5575 leader_election.cc:304] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e; no voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:50.014626  5879 raft_consensus.cc:3060] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:50.015915  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa" is_pre_election: true
I20260709 13:14:50.016261  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:50.016729  5683 raft_consensus.cc:2393] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 03fb445d9b604085af1649bdedf5f55e in current term 1: Already voted for candidate 61b1acaac8bf494cb41df91d5c6affca in this term.
I20260709 13:14:50.017859  5924 leader_election.cc:290] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:50.017930  5575 leader_election.cc:304] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e; no voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:50.019177  5924 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:50.022140  5879 raft_consensus.cc:2749] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.021988  5927 consensus_queue.cc:1048] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.028697  5924 raft_consensus.cc:2749] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.037323  5925 consensus_queue.cc:1048] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:50.038847  5880 raft_consensus.cc:493] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.039390  5880 raft_consensus.cc:515] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.041399  5880 leader_election.cc:290] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:50.043339  5757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b1acaac8bf494cb41df91d5c6affca" is_pre_election: true
I20260709 13:14:50.046268  5608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03fb445d9b604085af1649bdedf5f55e" is_pre_election: true
I20260709 13:14:50.047006  5608 raft_consensus.cc:2393] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 204d6918cdaf414f9860e5cbc718b0aa in current term 1: Already voted for candidate 61b1acaac8bf494cb41df91d5c6affca in this term.
I20260709 13:14:50.047995  5647 leader_election.cc:304] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa; no voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:50.049616  5880 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:50.058846  5880 raft_consensus.cc:2749] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.088989  5756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "03fb445d9b604085af1649bdedf5f55e"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:50.089709  5756 raft_consensus.cc:606] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:50.094967  5756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aca642f543974064bbacc6639ad410ac"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "03fb445d9b604085af1649bdedf5f55e"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:50.095419  5756 raft_consensus.cc:606] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:50.102486  5756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:50.102927  5756 raft_consensus.cc:606] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:50.107442  5756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:50.107859  5756 raft_consensus.cc:606] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:50.112500  5756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
mode: GRACEFUL
new_leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:50412
I20260709 13:14:50.112920  5756 raft_consensus.cc:606] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Received request to transfer leadership to TS 204d6918cdaf414f9860e5cbc718b0aa
I20260709 13:14:50.127522  5683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f9eb71b83dd4911ac125e54e2ce8466"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
mode: GRACEFUL
new_leader_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:42082
I20260709 13:14:50.128145  5683 raft_consensus.cc:606] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Received request to transfer leadership to TS 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:50.144680  5915 consensus_queue.cc:1048] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.154109  5822 raft_consensus.cc:993] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 03fb445d9b604085af1649bdedf5f55e to start an election
I20260709 13:14:50.154615  5822 raft_consensus.cc:1081] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 03fb445d9b604085af1649bdedf5f55e to start an election
I20260709 13:14:50.158685  5934 consensus_queue.cc:1048] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.159266  5607 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654"
dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
 from {username='slave'} at 127.0.0.1:41434
I20260709 13:14:50.159888  5607 raft_consensus.cc:493] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.160274  5607 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.162094  5822 consensus_queue.cc:1048] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.173218  5607 raft_consensus.cc:515] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.176577  5607 leader_election.cc:290] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 2 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:50.180260  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:50.180879  5684 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.180891  5934 consensus_queue.cc:1048] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:14:50.188086  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:50.188688  5756 raft_consensus.cc:3055] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:50.188987  5756 raft_consensus.cc:740] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:50.189590  5684 raft_consensus.cc:2468] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 2.
I20260709 13:14:50.189857  5756 consensus_queue.cc:260] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.190945  5575 leader_election.cc:304] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:50.191455  5756 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.191742  5879 raft_consensus.cc:2804] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:50.192394  5879 raft_consensus.cc:697] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:50.193143  5879 consensus_queue.cc:237] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.198050  5931 raft_consensus.cc:993] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:50.198565  5935 raft_consensus.cc:1081] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:50.200145  5684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:42068
I20260709 13:14:50.200730  5684 raft_consensus.cc:493] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.201141  5684 raft_consensus.cc:3060] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.202270  5756 raft_consensus.cc:2468] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 2.
I20260709 13:14:50.202803  5931 consensus_queue.cc:1048] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.206300  5503 catalog_manager.cc:5697] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 2 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:50.208480  5684 raft_consensus.cc:515] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.211229  5608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:50.211501  5935 raft_consensus.cc:993] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:50.211782  5608 raft_consensus.cc:3060] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.211988  5935 raft_consensus.cc:1081] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:50.217927  5683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:42068
I20260709 13:14:50.218459  5683 raft_consensus.cc:493] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.218755  5683 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.219192  5931 consensus_queue.cc:1048] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.222826  5608 raft_consensus.cc:2468] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:50.224462  5647 leader_election.cc:304] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:50.225374  5683 raft_consensus.cc:515] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.226321  5929 raft_consensus.cc:2804] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:50.226542  5928 consensus_queue.cc:1048] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.226828  5929 raft_consensus.cc:697] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:50.227201  5683 leader_election.cc:290] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:50.227681  5929 consensus_queue.cc:237] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.231367  5684 leader_election.cc:290] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:50.234401  5609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:50.234484  5756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:50.235085  5756 raft_consensus.cc:3055] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:50.235381  5756 raft_consensus.cc:740] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:50.236217  5756 consensus_queue.cc:260] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.237200  5756 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.243763  5757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859cc219194843cbbe50c4e29eb66328" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:50.244526  5757 raft_consensus.cc:3055] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:50.244894  5757 raft_consensus.cc:740] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:50.244977  5756 raft_consensus.cc:2468] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:50.245826  5757 consensus_queue.cc:260] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.246861  5929 consensus_queue.cc:1048] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.247326  5757 raft_consensus.cc:3060] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.250221  5647 leader_election.cc:304] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 03fb445d9b604085af1649bdedf5f55e
I20260709 13:14:50.253147  5929 raft_consensus.cc:2804] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:50.253813  5929 raft_consensus.cc:697] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:50.254601  5929 consensus_queue.cc:237] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.264772  5757 raft_consensus.cc:2468] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:50.260476  5506 catalog_manager.cc:5697] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 2 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:50.281745  5503 catalog_manager.cc:5697] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 2 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:50.356575  5822 raft_consensus.cc:993] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 03fb445d9b604085af1649bdedf5f55e to start an election
I20260709 13:14:50.357076  5931 raft_consensus.cc:1081] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 03fb445d9b604085af1649bdedf5f55e to start an election
I20260709 13:14:50.358660  5607 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aca642f543974064bbacc6639ad410ac"
dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
 from {username='slave'} at 127.0.0.1:41434
I20260709 13:14:50.359201  5607 raft_consensus.cc:493] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.359583  5607 raft_consensus.cc:3060] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.364686  5607 raft_consensus.cc:515] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.366204  5607 leader_election.cc:290] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 2 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:50.368333  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca642f543974064bbacc6639ad410ac" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:50.368067  5684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca642f543974064bbacc6639ad410ac" candidate_uuid: "03fb445d9b604085af1649bdedf5f55e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:50.369022  5684 raft_consensus.cc:3060] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.370828  5758 raft_consensus.cc:3055] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:50.371270  5758 raft_consensus.cc:740] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:50.372184  5758 consensus_queue.cc:260] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.373442  5758 raft_consensus.cc:3060] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.376700  5684 raft_consensus.cc:2468] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 2.
I20260709 13:14:50.377820  5575 leader_election.cc:304] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:50.378588  5879 raft_consensus.cc:2804] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:50.379014  5879 raft_consensus.cc:697] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Becoming Leader. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:50.379175  5758 raft_consensus.cc:2468] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03fb445d9b604085af1649bdedf5f55e in term 2.
I20260709 13:14:50.379798  5879 consensus_queue.cc:237] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.385001  5931 consensus_queue.cc:1048] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.387681  5503 catalog_manager.cc:5697] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 03fb445d9b604085af1649bdedf5f55e (127.27.63.65). New cstate: current_term: 2 leader_uuid: "03fb445d9b604085af1649bdedf5f55e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:50.394680  5931 consensus_queue.cc:1048] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.405041  5934 raft_consensus.cc:993] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: : Instructing follower 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:50.405544  5931 raft_consensus.cc:1081] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Signalling peer 204d6918cdaf414f9860e5cbc718b0aa to start an election
I20260709 13:14:50.406998  5684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
 from {username='slave'} at 127.0.0.1:42068
I20260709 13:14:50.407534  5684 raft_consensus.cc:493] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.407887  5684 raft_consensus.cc:3060] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.412626  5684 raft_consensus.cc:515] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.414381  5684 leader_election.cc:290] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67:36081)
I20260709 13:14:50.415467  5758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
I20260709 13:14:50.415915  5607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05634e54fd08425ea500f4f4a0dc145d" candidate_uuid: "204d6918cdaf414f9860e5cbc718b0aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:50.416146  5758 raft_consensus.cc:3055] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:50.416493  5758 raft_consensus.cc:740] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:50.416513  5607 raft_consensus.cc:3060] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.417361  5758 consensus_queue.cc:260] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.418680  5758 raft_consensus.cc:3060] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.424304  5607 raft_consensus.cc:2468] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:50.425465  5647 leader_election.cc:304] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 204d6918cdaf414f9860e5cbc718b0aa; no voters: 
I20260709 13:14:50.425580  5758 raft_consensus.cc:2468] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204d6918cdaf414f9860e5cbc718b0aa in term 2.
I20260709 13:14:50.426251  5929 raft_consensus.cc:2804] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:50.426753  5929 raft_consensus.cc:697] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Becoming Leader. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:50.427502  5929 consensus_queue.cc:237] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.433283  5505 catalog_manager.cc:5697] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa reported cstate change: term changed from 1 to 2, leader changed from 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67) to 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66). New cstate: current_term: 2 leader_uuid: "204d6918cdaf414f9860e5cbc718b0aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:50.506426  5966 consensus_queue.cc:1048] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.513074  5929 raft_consensus.cc:993] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: : Instructing follower 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:50.513643  5928 raft_consensus.cc:1081] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Signalling peer 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:50.516762  5757 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f9eb71b83dd4911ac125e54e2ce8466"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:50402
I20260709 13:14:50.517343  5757 raft_consensus.cc:493] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.517704  5757 raft_consensus.cc:3060] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.519415  5928 consensus_queue.cc:1048] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:50.526203  5757 raft_consensus.cc:515] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.527331  5758 raft_consensus.cc:1240] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Rejecting Update request from peer 204d6918cdaf414f9860e5cbc718b0aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:50.528644  5757 leader_election.cc:290] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 2 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:50.529217  5607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f9eb71b83dd4911ac125e54e2ce8466" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:50.529812  5607 raft_consensus.cc:3060] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.531061  5683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f9eb71b83dd4911ac125e54e2ce8466" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:50.531629  5683 raft_consensus.cc:3055] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:50.531921  5683 raft_consensus.cc:740] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:50.532631  5683 consensus_queue.cc:260] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.533625  5683 raft_consensus.cc:3060] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:50.534006  5928 consensus_queue.cc:1175] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "61b1acaac8bf494cb41df91d5c6affca" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 0 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 204d6918cdaf414f9860e5cbc718b0aa for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:50.537846  5607 raft_consensus.cc:2468] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 2.
I20260709 13:14:50.539129  5721 leader_election.cc:304] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:50.539937  5931 raft_consensus.cc:2804] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:50.540275  5683 raft_consensus.cc:2468] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 2.
I20260709 13:14:50.540469  5931 raft_consensus.cc:697] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 2 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:50.541337  5931 consensus_queue.cc:237] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:50.548727  5505 catalog_manager.cc:5697] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 1 to 2, leader changed from 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66) to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 2 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:50.670506  5683 raft_consensus.cc:1275] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Refusing update from remote peer 03fb445d9b604085af1649bdedf5f55e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.671790  5879 consensus_queue.cc:1048] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.678813  5758 raft_consensus.cc:1275] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 03fb445d9b604085af1649bdedf5f55e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.681216  5879 consensus_queue.cc:1048] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.719826  5758 raft_consensus.cc:1275] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.721225  5928 consensus_queue.cc:1048] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.729452  5608 raft_consensus.cc:1275] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.730741  5607 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.731297  5928 consensus_queue.cc:1048] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.738129  5607 raft_consensus.cc:1275] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.739748  5758 raft_consensus.cc:1275] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.741364  5880 consensus_queue.cc:1048] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.741426  5928 consensus_queue.cc:1048] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.812269  5683 raft_consensus.cc:1275] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Refusing update from remote peer 03fb445d9b604085af1649bdedf5f55e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.813426  5879 consensus_queue.cc:1048] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.820322  5757 raft_consensus.cc:1275] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 03fb445d9b604085af1649bdedf5f55e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.822045  5927 consensus_queue.cc:1048] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.917758  5757 raft_consensus.cc:1275] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.919092  5880 consensus_queue.cc:1048] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:50.927629  5608 raft_consensus.cc:1275] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Refusing update from remote peer 204d6918cdaf414f9860e5cbc718b0aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:50.929220  5929 consensus_queue.cc:1048] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.008596  5608 raft_consensus.cc:1275] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:51.009976  5931 consensus_queue.cc:1048] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.019431  5683 raft_consensus.cc:1275] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:51.021462  5973 consensus_queue.cc:1048] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:53.132488  5503 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47618:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:14:53.160391  5503 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47618:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:14:53.291486 27901 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:14:53.391685  5993 log.cc:927] Time spent T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: Append to log took a long time: real 0.130s	user 0.002s	sys 0.000s
I20260709 13:14:53.452757  6011 tablet.cc:1724] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:14:53.462311  6003 log.cc:927] Time spent T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:14:53.534833  6018 tablet.cc:1724] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:14:53.586521  5784 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:04df0d979cc246009e861b3bcb62016f)
I20260709 13:14:53.586814  6030 tablet.cc:1724] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:14:53.591604  5784 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:04df0d979cc246009e861b3bcb62016f)
W20260709 13:14:53.593293  5784 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:04df0d979cc246009e861b3bcb62016f)
I20260709 13:14:53.600365  6034 tablet.cc:1724] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:14:53.610090  6024 log.cc:927] Time spent T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:14:53.616919  6038 tablet.cc:1724] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:14:53.629837  6020 log.cc:927] Time spent T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e: Append to log took a long time: real 0.157s	user 0.003s	sys 0.000s
I20260709 13:14:53.712241  6041 tablet.cc:1724] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.737241  6044 tablet.cc:1724] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:14:53.739446  5721 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.65:46869, user_credentials={real_user=slave}} blocked reactor thread for 44539.8us
I20260709 13:14:53.742612  6018 tablet.cc:1724] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:14:53.762605  5647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.63.67:36081, user_credentials={real_user=slave}} blocked reactor thread for 80816.7us
I20260709 13:14:53.763625  5681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
mode: GRACEFUL
new_leader_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:38564
I20260709 13:14:53.774567  6030 tablet.cc:1724] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.764716  5681 raft_consensus.cc:606] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Received request to transfer leadership to TS 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:53.785653  6046 tablet.cc:1724] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.800695  6011 tablet.cc:1724] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.807801  6047 tablet.cc:1724] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.811465  5880 raft_consensus.cc:993] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: : Instructing follower 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:53.812070  5880 raft_consensus.cc:1081] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Signalling peer 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:53.720692  6040 tablet.cc:1724] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.816624  6049 tablet.cc:1724] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.818502  5758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:50402
I20260709 13:14:53.819260  5758 raft_consensus.cc:493] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:53.819769  5758 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:53.820577  6038 tablet.cc:1724] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.832767  6048 tablet.cc:1724] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.833411  6041 tablet.cc:1724] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.848003  6034 tablet.cc:1724] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.830936  6046 tablet.cc:1724] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.869642  6011 tablet.cc:1724] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.872371  6044 tablet.cc:1724] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.875773  6040 tablet.cc:1724] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.876911  6049 tablet.cc:1724] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.859424  5758 raft_consensus.cc:515] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:53.885496  5679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:53.885594  5603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3f7439b0f949debb699c92a65c3df5" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:53.886233  5679 raft_consensus.cc:3055] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:53.886348  5603 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:53.886619  5679 raft_consensus.cc:740] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:53.887693  5679 consensus_queue.cc:260] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:53.888947  5679 raft_consensus.cc:3060] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:53.888808  6047 tablet.cc:1724] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.904453  6018 tablet.cc:1724] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.915059  5603 raft_consensus.cc:2468] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 3.
I20260709 13:14:53.904822  6048 tablet.cc:1724] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.908702  6038 tablet.cc:1724] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:14:53.905193  5679 raft_consensus.cc:2468] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 3.
I20260709 13:14:53.922348  5724 leader_election.cc:304] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:53.923995  6045 raft_consensus.cc:2804] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:53.924567  6045 raft_consensus.cc:697] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 3 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:53.925704  6045 consensus_queue.cc:237] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:53.928216  5607 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654"
dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
mode: GRACEFUL
new_leader_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:44712
I20260709 13:14:53.928891  5607 raft_consensus.cc:606] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Received request to transfer leadership to TS 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:53.930958 27901 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:14:53.934255 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:53.946601  5758 leader_election.cc:290] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 3 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:53.949024  5607 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2"
dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
mode: GRACEFUL
new_leader_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:44712
I20260709 13:14:53.949673  5607 raft_consensus.cc:606] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Received request to transfer leadership to TS 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:53.955327  5679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde"
dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
mode: GRACEFUL
new_leader_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:38564
I20260709 13:14:53.955973  5679 raft_consensus.cc:606] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Received request to transfer leadership to TS 61b1acaac8bf494cb41df91d5c6affca
I20260709 13:14:53.957872 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:14:53.958461 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:53.962849  5505 catalog_manager.cc:5697] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 2 to 3, leader changed from 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66) to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 3 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:53.978794  5503 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b: soft_deleted_table [id=037b99722ae94433a4d5d7cebf83380a] alter complete (version 1)
I20260709 13:14:54.175213  6016 raft_consensus.cc:993] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e: : Instructing follower 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:54.175805  5977 raft_consensus.cc:1081] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Signalling peer 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:54.177539  5758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:50398
I20260709 13:14:54.178033  5758 raft_consensus.cc:493] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:54.178426  5758 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:54.184871  5758 raft_consensus.cc:515] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.187188  5607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:54.187423  5679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bada308d1e64b90adbc36c78c4d0654" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:54.187839  5607 raft_consensus.cc:3055] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:54.188066  5679 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:54.188259  5607 raft_consensus.cc:740] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:54.189193  5607 consensus_queue.cc:260] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.190394  5607 raft_consensus.cc:3060] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:54.194167  5758 leader_election.cc:290] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 3 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:54.195549  5679 raft_consensus.cc:2468] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 3.
I20260709 13:14:54.196816  5724 leader_election.cc:304] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:54.197530  6045 raft_consensus.cc:2804] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:54.197953  6045 raft_consensus.cc:697] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 3 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:54.198855  5607 raft_consensus.cc:2468] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 3.
I20260709 13:14:54.198748  6045 consensus_queue.cc:237] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.205689  5505 catalog_manager.cc:5697] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 2 to 3, leader changed from 03fb445d9b604085af1649bdedf5f55e (127.27.63.65) to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 3 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:54.221336  6043 raft_consensus.cc:993] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa: : Instructing follower 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:54.221990  5978 raft_consensus.cc:1081] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Signalling peer 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:54.223374  5758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:50402
I20260709 13:14:54.223743  5758 raft_consensus.cc:493] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:54.223958  5758 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:54.228739  5758 raft_consensus.cc:515] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.230232  5758 leader_election.cc:290] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 3 election: Requested vote from peers 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869), 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993)
I20260709 13:14:54.230994  5607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:54.231353  5679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5284f1c6b1f24d938f03bebbdf7b3bde" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:54.231738  5607 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:54.232065  5679 raft_consensus.cc:3055] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:54.232445  5679 raft_consensus.cc:740] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 204d6918cdaf414f9860e5cbc718b0aa, State: Running, Role: LEADER
I20260709 13:14:54.233330  5679 consensus_queue.cc:260] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.234378  5679 raft_consensus.cc:3060] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:54.237119  5607 raft_consensus.cc:2468] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 3.
I20260709 13:14:54.238240  5721 leader_election.cc:304] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03fb445d9b604085af1649bdedf5f55e, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:54.238943  6045 raft_consensus.cc:2804] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:54.239341  5679 raft_consensus.cc:2468] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 3.
I20260709 13:14:54.239354  6045 raft_consensus.cc:697] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 3 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:54.240485  6045 consensus_queue.cc:237] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.247483  5505 catalog_manager.cc:5697] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 2 to 3, leader changed from 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66) to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 3 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:54.346225  5977 raft_consensus.cc:993] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e: : Instructing follower 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:54.346779  6016 raft_consensus.cc:1081] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Signalling peer 61b1acaac8bf494cb41df91d5c6affca to start an election
I20260709 13:14:54.348404  5758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2"
dest_uuid: "61b1acaac8bf494cb41df91d5c6affca"
 from {username='slave'} at 127.0.0.1:50398
I20260709 13:14:54.348892  5758 raft_consensus.cc:493] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:54.349202  5758 raft_consensus.cc:3060] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.355810  5758 raft_consensus.cc:515] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.357208  5758 leader_election.cc:290] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 2 election: Requested vote from peers 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993), 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869)
I20260709 13:14:54.358022  5679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "204d6918cdaf414f9860e5cbc718b0aa"
I20260709 13:14:54.358353  5607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03158be0aa71421d89031e4b7fad6af2" candidate_uuid: "61b1acaac8bf494cb41df91d5c6affca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03fb445d9b604085af1649bdedf5f55e"
I20260709 13:14:54.358613  5679 raft_consensus.cc:3060] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.359004  5607 raft_consensus.cc:3055] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:54.359314  5607 raft_consensus.cc:740] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 03fb445d9b604085af1649bdedf5f55e, State: Running, Role: LEADER
I20260709 13:14:54.360193  5607 consensus_queue.cc:260] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.361404  5607 raft_consensus.cc:3060] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.363734  5679 raft_consensus.cc:2468] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 2.
I20260709 13:14:54.364805  5724 leader_election.cc:304] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204d6918cdaf414f9860e5cbc718b0aa, 61b1acaac8bf494cb41df91d5c6affca; no voters: 
I20260709 13:14:54.365502  6045 raft_consensus.cc:2804] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:54.365938  6045 raft_consensus.cc:697] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 2 LEADER]: Becoming Leader. State: Replica: 61b1acaac8bf494cb41df91d5c6affca, State: Running, Role: LEADER
I20260709 13:14:54.366895  6045 consensus_queue.cc:237] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } }
I20260709 13:14:54.368309  5607 raft_consensus.cc:2468] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b1acaac8bf494cb41df91d5c6affca in term 2.
I20260709 13:14:54.377146  5505 catalog_manager.cc:5697] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca reported cstate change: term changed from 1 to 2, leader changed from 03fb445d9b604085af1649bdedf5f55e (127.27.63.65) to 61b1acaac8bf494cb41df91d5c6affca (127.27.63.67). New cstate: current_term: 2 leader_uuid: "61b1acaac8bf494cb41df91d5c6affca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b1acaac8bf494cb41df91d5c6affca" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 36081 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:54.512266  5607 raft_consensus.cc:1275] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:14:54.513595  6045 consensus_queue.cc:1048] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:54.522583  5679 raft_consensus.cc:1275] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:14:54.524144  6045 consensus_queue.cc:1048] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:54.648244  5679 raft_consensus.cc:1275] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:54.648682  5680 raft_consensus.cc:1275] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:14:54.649744  6045 consensus_queue.cc:1048] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:54.650180  6039 consensus_queue.cc:1048] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:54.672324  5603 raft_consensus.cc:1275] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:54.674753  5603 raft_consensus.cc:1275] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:14:54.675637  6039 consensus_queue.cc:1048] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:54.676656  6045 consensus_queue.cc:1048] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:54.836517  5680 raft_consensus.cc:1275] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:54.837769  6062 consensus_queue.cc:1048] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "204d6918cdaf414f9860e5cbc718b0aa" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.847204  5603 raft_consensus.cc:1275] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Refusing update from remote peer 61b1acaac8bf494cb41df91d5c6affca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:54.848995  6045 consensus_queue.cc:1048] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [LEADER]: Connected to new peer: Peer: permanent_uuid: "03fb445d9b604085af1649bdedf5f55e" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.014058 27901 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:14:55.018953 27901 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:14:55.019397 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:55.024766 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:14:55.025326 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:56.054386 27901 tablet_server.cc:179] TabletServer@127.27.63.65:0 shutting down...
I20260709 13:14:56.080530 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:14:56.080667  5721 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
I20260709 13:14:56.081565 27901 tablet_replica.cc:333] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.082834 27901 raft_consensus.cc:2243] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.083480 27901 raft_consensus.cc:2272] T 2f9eb71b83dd4911ac125e54e2ce8466 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:56.085987  5721 consensus_peers.cc:597] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca -> Peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Couldn't send request to peer 03fb445d9b604085af1649bdedf5f55e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:56.086205 27901 tablet_replica.cc:333] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.087081 27901 raft_consensus.cc:2243] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.087572 27901 raft_consensus.cc:2272] T 859cc219194843cbbe50c4e29eb66328 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.090306 27901 tablet_replica.cc:333] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.091080 27901 raft_consensus.cc:2243] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.091526 27901 raft_consensus.cc:2272] T 8bada308d1e64b90adbc36c78c4d0654 P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.093565 27901 tablet_replica.cc:333] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.094164 27901 raft_consensus.cc:2243] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.094871 27901 raft_consensus.cc:2272] T aca642f543974064bbacc6639ad410ac P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.096845 27901 tablet_replica.cc:333] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.097401 27901 raft_consensus.cc:2243] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.097828 27901 raft_consensus.cc:2272] T 05634e54fd08425ea500f4f4a0dc145d P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.099761 27901 tablet_replica.cc:333] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.100395 27901 raft_consensus.cc:2243] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.100845 27901 raft_consensus.cc:2272] T 7e3f7439b0f949debb699c92a65c3df5 P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.102919 27901 tablet_replica.cc:333] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.103456 27901 raft_consensus.cc:2243] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.103856 27901 raft_consensus.cc:2272] T 59bd46f0367d4b96ab704bb1a32ab93d P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.105819 27901 tablet_replica.cc:333] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.106400 27901 raft_consensus.cc:2243] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.106840 27901 raft_consensus.cc:2272] T 91ba4865f65a49a093a45bacdee5f8cd P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.108879 27901 tablet_replica.cc:333] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.109454 27901 raft_consensus.cc:2243] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.109895 27901 raft_consensus.cc:2272] T 1233b88a15c64dea8fea255c68f3b264 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.112074 27901 tablet_replica.cc:333] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.112782 27901 raft_consensus.cc:2243] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.113337 27901 raft_consensus.cc:2272] T 5bfad76b6e004e95bc768e524eaa3b62 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.115551 27901 tablet_replica.cc:333] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.116158 27901 raft_consensus.cc:2243] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.116977 27901 raft_consensus.cc:2272] T 5dce9f9c479c4d7a95cc75044ece77a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.118949 27901 tablet_replica.cc:333] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.119524 27901 raft_consensus.cc:2243] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.120292 27901 raft_consensus.cc:2272] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.122321 27901 tablet_replica.cc:333] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.122845 27901 raft_consensus.cc:2243] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.123256 27901 raft_consensus.cc:2272] T 04df0d979cc246009e861b3bcb62016f P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.125413 27901 tablet_replica.cc:333] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.125921 27901 raft_consensus.cc:2243] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.126451 27901 raft_consensus.cc:2272] T 5284f1c6b1f24d938f03bebbdf7b3bde P 03fb445d9b604085af1649bdedf5f55e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.128580 27901 tablet_replica.cc:333] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.129181 27901 raft_consensus.cc:2243] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.130051 27901 raft_consensus.cc:2272] T 924c6b951ad34e9f9cc3610764e2f7c3 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.132098 27901 tablet_replica.cc:333] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.132608 27901 raft_consensus.cc:2243] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.133366 27901 raft_consensus.cc:2272] T 9f59faadac8a40caad4aead074d3ad1f P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.135227 27901 tablet_replica.cc:333] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.135725 27901 raft_consensus.cc:2243] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.136505 27901 raft_consensus.cc:2272] T 09345849992a47948144e243e591e4ea P 03fb445d9b604085af1649bdedf5f55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.138396 27901 tablet_replica.cc:333] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e: stopping tablet replica
I20260709 13:14:56.138847 27901 raft_consensus.cc:2243] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.139238 27901 raft_consensus.cc:2272] T 03158be0aa71421d89031e4b7fad6af2 P 03fb445d9b604085af1649bdedf5f55e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:56.161355  5647 consensus_peers.cc:597] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa -> Peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Couldn't send request to peer 03fb445d9b604085af1649bdedf5f55e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:56.176963  5721 consensus_peers.cc:597] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca -> Peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Couldn't send request to peer 03fb445d9b604085af1649bdedf5f55e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:46869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:56.183061 27901 tablet_server.cc:196] TabletServer@127.27.63.65:0 shutdown complete.
W20260709 13:14:56.199178  5647 consensus_peers.cc:597] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa -> Peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Couldn't send request to peer 03fb445d9b604085af1649bdedf5f55e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:46869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:56.219663 27901 tablet_server.cc:179] TabletServer@127.27.63.66:0 shutting down...
W20260709 13:14:56.223438  5647 consensus_peers.cc:597] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa -> Peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Couldn't send request to peer 03fb445d9b604085af1649bdedf5f55e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:46869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:56.230511  5724 consensus_peers.cc:597] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca -> Peer 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993): Couldn't send request to peer 204d6918cdaf414f9860e5cbc718b0aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:56.249275 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:56.250074 27901 tablet_replica.cc:333] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.250762 27901 raft_consensus.cc:2243] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.251260 27901 raft_consensus.cc:2272] T 7e3f7439b0f949debb699c92a65c3df5 P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.253434 27901 tablet_replica.cc:333] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.254043 27901 raft_consensus.cc:2243] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.254503 27901 raft_consensus.cc:2272] T 2f9eb71b83dd4911ac125e54e2ce8466 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.257257 27901 tablet_replica.cc:333] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.257786 27901 raft_consensus.cc:2243] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.258208 27901 raft_consensus.cc:2272] T 8bada308d1e64b90adbc36c78c4d0654 P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.260004 27901 tablet_replica.cc:333] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.260591 27901 raft_consensus.cc:2243] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.261341 27901 raft_consensus.cc:2272] T 859cc219194843cbbe50c4e29eb66328 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.263250 27901 tablet_replica.cc:333] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.263737 27901 raft_consensus.cc:2243] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.264240 27901 raft_consensus.cc:2272] T 5dce9f9c479c4d7a95cc75044ece77a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.266005 27901 tablet_replica.cc:333] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.266572 27901 raft_consensus.cc:2243] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.267295 27901 raft_consensus.cc:2272] T 05634e54fd08425ea500f4f4a0dc145d P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.269277 27901 tablet_replica.cc:333] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.269804 27901 raft_consensus.cc:2243] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.270221 27901 raft_consensus.cc:2272] T aca642f543974064bbacc6639ad410ac P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.272142 27901 tablet_replica.cc:333] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.272686 27901 raft_consensus.cc:2243] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.273445 27901 raft_consensus.cc:2272] T 1233b88a15c64dea8fea255c68f3b264 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.275440 27901 tablet_replica.cc:333] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.275995 27901 raft_consensus.cc:2243] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.276823 27901 raft_consensus.cc:2272] T 5bfad76b6e004e95bc768e524eaa3b62 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.278877 27901 tablet_replica.cc:333] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.279424 27901 raft_consensus.cc:2243] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.280385 27901 raft_consensus.cc:2272] T 91ba4865f65a49a093a45bacdee5f8cd P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.283005 27901 tablet_replica.cc:333] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.283578 27901 raft_consensus.cc:2243] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.284015 27901 raft_consensus.cc:2272] T 59bd46f0367d4b96ab704bb1a32ab93d P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.286037 27901 tablet_replica.cc:333] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.286670 27901 raft_consensus.cc:2243] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.287101 27901 raft_consensus.cc:2272] T 03158be0aa71421d89031e4b7fad6af2 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.289053 27901 tablet_replica.cc:333] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.289542 27901 raft_consensus.cc:2243] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.289944 27901 raft_consensus.cc:2272] T 09345849992a47948144e243e591e4ea P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.291821 27901 tablet_replica.cc:333] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.292397 27901 raft_consensus.cc:2243] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.292820 27901 raft_consensus.cc:2272] T 9f59faadac8a40caad4aead074d3ad1f P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.294727 27901 tablet_replica.cc:333] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.295219 27901 raft_consensus.cc:2243] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.295622 27901 raft_consensus.cc:2272] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.297502 27901 tablet_replica.cc:333] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.297998 27901 raft_consensus.cc:2243] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.298734 27901 raft_consensus.cc:2272] T 04df0d979cc246009e861b3bcb62016f P 204d6918cdaf414f9860e5cbc718b0aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.300653 27901 tablet_replica.cc:333] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.301138 27901 raft_consensus.cc:2243] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.301530 27901 raft_consensus.cc:2272] T 5284f1c6b1f24d938f03bebbdf7b3bde P 204d6918cdaf414f9860e5cbc718b0aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.303385 27901 tablet_replica.cc:333] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa: stopping tablet replica
I20260709 13:14:56.303895 27901 raft_consensus.cc:2243] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.304353 27901 raft_consensus.cc:2272] T 924c6b951ad34e9f9cc3610764e2f7c3 P 204d6918cdaf414f9860e5cbc718b0aa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:56.335275  5721 consensus_peers.cc:597] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca -> Peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Couldn't send request to peer 03fb445d9b604085af1649bdedf5f55e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:46869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:56.344622 27901 tablet_server.cc:196] TabletServer@127.27.63.66:0 shutdown complete.
W20260709 13:14:56.364549  5721 consensus_peers.cc:597] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca -> Peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Couldn't send request to peer 03fb445d9b604085af1649bdedf5f55e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:46869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:56.377030  5724 consensus_peers.cc:597] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca -> Peer 204d6918cdaf414f9860e5cbc718b0aa (127.27.63.66:36993): Couldn't send request to peer 204d6918cdaf414f9860e5cbc718b0aa. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.66:36993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:56.384549 27901 tablet_server.cc:179] TabletServer@127.27.63.67:0 shutting down...
W20260709 13:14:56.400487  5721 consensus_peers.cc:597] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca -> Peer 03fb445d9b604085af1649bdedf5f55e (127.27.63.65:46869): Couldn't send request to peer 03fb445d9b604085af1649bdedf5f55e. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:46869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:56.413931 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:56.414719 27901 tablet_replica.cc:333] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.415716 27901 raft_consensus.cc:2243] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.416711 27901 raft_consensus.cc:2272] T 2f9eb71b83dd4911ac125e54e2ce8466 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.418807 27901 tablet_replica.cc:333] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.419307 27901 raft_consensus.cc:2243] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.419695 27901 raft_consensus.cc:2272] T 13a4ca4d476d4a7fb07defd2dfcb67a7 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.421520 27901 tablet_replica.cc:333] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.422000 27901 raft_consensus.cc:2243] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.422662 27901 raft_consensus.cc:2272] T 8bada308d1e64b90adbc36c78c4d0654 P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.425278 27901 tablet_replica.cc:333] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.426124 27901 raft_consensus.cc:2243] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.426748 27901 raft_consensus.cc:2272] T 859cc219194843cbbe50c4e29eb66328 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.429373 27901 tablet_replica.cc:333] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.429965 27901 raft_consensus.cc:2243] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.430429 27901 raft_consensus.cc:2272] T 05634e54fd08425ea500f4f4a0dc145d P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.432735 27901 tablet_replica.cc:333] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.433274 27901 raft_consensus.cc:2243] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.433661 27901 raft_consensus.cc:2272] T aca642f543974064bbacc6639ad410ac P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.435400 27901 tablet_replica.cc:333] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.435904 27901 raft_consensus.cc:2243] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.436630 27901 raft_consensus.cc:2272] T 7e3f7439b0f949debb699c92a65c3df5 P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.438436 27901 tablet_replica.cc:333] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.438925 27901 raft_consensus.cc:2243] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.439268 27901 raft_consensus.cc:2272] T 09345849992a47948144e243e591e4ea P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.442147 27901 tablet_replica.cc:333] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.442641 27901 raft_consensus.cc:2243] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.443282 27901 raft_consensus.cc:2272] T 03158be0aa71421d89031e4b7fad6af2 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.445245 27901 tablet_replica.cc:333] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.445746 27901 raft_consensus.cc:2243] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.446084 27901 raft_consensus.cc:2272] T 91ba4865f65a49a093a45bacdee5f8cd P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.447933 27901 tablet_replica.cc:333] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.448560 27901 raft_consensus.cc:2243] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.449393 27901 raft_consensus.cc:2272] T 59bd46f0367d4b96ab704bb1a32ab93d P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.451249 27901 tablet_replica.cc:333] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.451709 27901 raft_consensus.cc:2243] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.452144 27901 raft_consensus.cc:2272] T 9f59faadac8a40caad4aead074d3ad1f P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.453948 27901 tablet_replica.cc:333] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.454391 27901 raft_consensus.cc:2243] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.454838 27901 raft_consensus.cc:2272] T 5dce9f9c479c4d7a95cc75044ece77a7 P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.456732 27901 tablet_replica.cc:333] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.457170 27901 raft_consensus.cc:2243] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.457530 27901 raft_consensus.cc:2272] T 04df0d979cc246009e861b3bcb62016f P 61b1acaac8bf494cb41df91d5c6affca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.459259 27901 tablet_replica.cc:333] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.459724 27901 raft_consensus.cc:2243] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.460505 27901 raft_consensus.cc:2272] T 5284f1c6b1f24d938f03bebbdf7b3bde P 61b1acaac8bf494cb41df91d5c6affca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.463007 27901 tablet_replica.cc:333] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.463690 27901 raft_consensus.cc:2243] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.464135 27901 raft_consensus.cc:2272] T 924c6b951ad34e9f9cc3610764e2f7c3 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.465867 27901 tablet_replica.cc:333] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.466280 27901 raft_consensus.cc:2243] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.466662 27901 raft_consensus.cc:2272] T 1233b88a15c64dea8fea255c68f3b264 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.468478 27901 tablet_replica.cc:333] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca: stopping tablet replica
I20260709 13:14:56.468858 27901 raft_consensus.cc:2243] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.469215 27901 raft_consensus.cc:2272] T 5bfad76b6e004e95bc768e524eaa3b62 P 61b1acaac8bf494cb41df91d5c6affca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.507292 27901 tablet_server.cc:196] TabletServer@127.27.63.67:0 shutdown complete.
I20260709 13:14:56.542224 27901 master.cc:562] Master@127.27.63.126:46611 shutting down...
I20260709 13:14:56.568239 27901 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:56.568768 27901 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.569047 27901 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2cf7de9bb6ed4420b26939d4cde7325b: stopping tablet replica
I20260709 13:14:56.588904 27901 master.cc:584] Master@127.27.63.126:46611 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12510 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:14:56.639020 27901 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.63.126:33459
I20260709 13:14:56.640240 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:56.645623  6075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:56.645759  6076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:56.647272  6078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:56.648183 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:56.649055 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:56.649243 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:56.649358 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602896649347 us; error 0 us; skew 500 ppm
I20260709 13:14:56.649802 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:56.652218 27901 webserver.cc:533] Webserver started at http://127.27.63.126:37365/ using document root <none> and password file <none>
I20260709 13:14:56.652668 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:56.652820 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:56.653033 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:56.654071 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/master-0-root/instance:
uuid: "db506e36d4b64a448c8a8a7dba0ee3f2"
format_stamp: "Formatted at 2026-07-09 13:14:56 on dist-test-slave-sjgx"
I20260709 13:14:56.658394 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:14:56.661685  6083 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:56.662454 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:14:56.662791 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/master-0-root
uuid: "db506e36d4b64a448c8a8a7dba0ee3f2"
format_stamp: "Formatted at 2026-07-09 13:14:56 on dist-test-slave-sjgx"
I20260709 13:14:56.663039 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:56.678061 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:56.679293 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:56.716290 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.126:33459
I20260709 13:14:56.716499  6134 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.126:33459 every 8 connection(s)
I20260709 13:14:56.720352  6135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:56.731787  6135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2: Bootstrap starting.
I20260709 13:14:56.737423  6135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:56.741909  6135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2: No bootstrap required, opened a new log
I20260709 13:14:56.744177  6135 raft_consensus.cc:359] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db506e36d4b64a448c8a8a7dba0ee3f2" member_type: VOTER }
I20260709 13:14:56.744590  6135 raft_consensus.cc:385] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:56.744788  6135 raft_consensus.cc:740] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db506e36d4b64a448c8a8a7dba0ee3f2, State: Initialized, Role: FOLLOWER
I20260709 13:14:56.745297  6135 consensus_queue.cc:260] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db506e36d4b64a448c8a8a7dba0ee3f2" member_type: VOTER }
I20260709 13:14:56.745774  6135 raft_consensus.cc:399] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:56.745994  6135 raft_consensus.cc:493] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:56.746222  6135 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:56.751550  6135 raft_consensus.cc:515] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db506e36d4b64a448c8a8a7dba0ee3f2" member_type: VOTER }
I20260709 13:14:56.752069  6135 leader_election.cc:304] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db506e36d4b64a448c8a8a7dba0ee3f2; no voters: 
I20260709 13:14:56.753227  6135 leader_election.cc:290] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:56.753613  6138 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:56.754979  6138 raft_consensus.cc:697] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 1 LEADER]: Becoming Leader. State: Replica: db506e36d4b64a448c8a8a7dba0ee3f2, State: Running, Role: LEADER
I20260709 13:14:56.755645  6138 consensus_queue.cc:237] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db506e36d4b64a448c8a8a7dba0ee3f2" member_type: VOTER }
I20260709 13:14:56.756484  6135 sys_catalog.cc:565] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:56.760114  6140 sys_catalog.cc:455] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db506e36d4b64a448c8a8a7dba0ee3f2. Latest consensus state: current_term: 1 leader_uuid: "db506e36d4b64a448c8a8a7dba0ee3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db506e36d4b64a448c8a8a7dba0ee3f2" member_type: VOTER } }
I20260709 13:14:56.760116  6139 sys_catalog.cc:455] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db506e36d4b64a448c8a8a7dba0ee3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db506e36d4b64a448c8a8a7dba0ee3f2" member_type: VOTER } }
I20260709 13:14:56.761222  6139 sys_catalog.cc:458] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:56.761236  6140 sys_catalog.cc:458] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:56.766155  6144 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:56.773260  6144 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:56.774263 27901 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:56.782701  6144 catalog_manager.cc:1383] Generated new cluster ID: f8d6afe7722843ab838f2cd84501c0a5
I20260709 13:14:56.782967  6144 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:56.802923  6144 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:56.804280  6144 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:56.822161  6144 catalog_manager.cc:6070] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2: Generated new TSK 0
I20260709 13:14:56.822855  6144 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:56.840780 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:56.846508  6156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:56.847894  6157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:56.849386  6159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:56.849865 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:56.850801 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:56.850975 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:56.851104 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602896851092 us; error 0 us; skew 500 ppm
I20260709 13:14:56.851521 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:56.853852 27901 webserver.cc:533] Webserver started at http://127.27.63.65:36601/ using document root <none> and password file <none>
I20260709 13:14:56.854286 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:56.854440 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:56.854646 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:56.855697 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-0-root/instance:
uuid: "16719ad7c0bd4042acd734beca09a9e0"
format_stamp: "Formatted at 2026-07-09 13:14:56 on dist-test-slave-sjgx"
I20260709 13:14:56.860289 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.002s
I20260709 13:14:56.863591  6164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:56.864498 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:14:56.864941 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-0-root
uuid: "16719ad7c0bd4042acd734beca09a9e0"
format_stamp: "Formatted at 2026-07-09 13:14:56 on dist-test-slave-sjgx"
I20260709 13:14:56.865260 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:56.901480 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:56.902765 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:56.904321 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:56.906571 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:56.906785 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:56.907027 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:56.907210 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:56.947739 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.65:33369
I20260709 13:14:56.947832  6226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.65:33369 every 8 connection(s)
I20260709 13:14:56.952819 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:56.959561  6231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:56.960497  6232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:56.962532 27901 server_base.cc:1061] running on GCE node
W20260709 13:14:56.964177  6234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:56.965257 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:14:56.965291  6227 heartbeater.cc:344] Connected to a master server at 127.27.63.126:33459
W20260709 13:14:56.965512 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:56.965776 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602896965759 us; error 0 us; skew 500 ppm
I20260709 13:14:56.965811  6227 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:56.966392 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:56.966763  6227 heartbeater.cc:507] Master 127.27.63.126:33459 requested a full tablet report, sending...
I20260709 13:14:56.969285  6100 ts_manager.cc:194] Registered new tserver with Master: 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369)
I20260709 13:14:56.969367 27901 webserver.cc:533] Webserver started at http://127.27.63.66:42225/ using document root <none> and password file <none>
I20260709 13:14:56.970068 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:56.970314 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:56.970628 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:56.971805  6100 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36306
I20260709 13:14:56.972213 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-1-root/instance:
uuid: "dd18efee76df4c2699bff236b9d056ba"
format_stamp: "Formatted at 2026-07-09 13:14:56 on dist-test-slave-sjgx"
I20260709 13:14:56.977059 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:14:56.980310  6239 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:56.981022 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:56.981326 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-1-root
uuid: "dd18efee76df4c2699bff236b9d056ba"
format_stamp: "Formatted at 2026-07-09 13:14:56 on dist-test-slave-sjgx"
I20260709 13:14:56.981585 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:56.999642 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:57.000914 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:57.002368 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:57.004570 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:57.004786 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:57.005020 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:57.005196 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:57.044881 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.66:36991
I20260709 13:14:57.045003  6301 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.66:36991 every 8 connection(s)
I20260709 13:14:57.049733 27901 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:57.056969  6305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:57.057716  6306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:57.062548  6302 heartbeater.cc:344] Connected to a master server at 127.27.63.126:33459
I20260709 13:14:57.063174  6302 heartbeater.cc:461] Registering TS with master...
W20260709 13:14:57.063380  6309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:57.064185  6302 heartbeater.cc:507] Master 127.27.63.126:33459 requested a full tablet report, sending...
I20260709 13:14:57.064692 27901 server_base.cc:1061] running on GCE node
I20260709 13:14:57.065737 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:57.065955 27901 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:57.066143 27901 hybrid_clock.cc:648] HybridClock initialized: now 1783602897066124 us; error 0 us; skew 500 ppm
I20260709 13:14:57.066619  6100 ts_manager.cc:194] Registered new tserver with Master: dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:57.066731 27901 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:57.068581  6100 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36320
I20260709 13:14:57.069753 27901 webserver.cc:533] Webserver started at http://127.27.63.67:35537/ using document root <none> and password file <none>
I20260709 13:14:57.070322 27901 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:57.070538 27901 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:57.070820 27901 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:57.071950 27901 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-2-root/instance:
uuid: "b3ae2edef6a0467eaf647236a2c51703"
format_stamp: "Formatted at 2026-07-09 13:14:57 on dist-test-slave-sjgx"
I20260709 13:14:57.076421 27901 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:57.079696  6313 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:57.080564 27901 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:57.080881 27901 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-2-root
uuid: "b3ae2edef6a0467eaf647236a2c51703"
format_stamp: "Formatted at 2026-07-09 13:14:57 on dist-test-slave-sjgx"
I20260709 13:14:57.081192 27901 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGsl_4k/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602591602838-27901-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:57.091159 27901 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:57.092361 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:57.093755 27901 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:57.095927 27901 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:57.096221 27901 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:14:57.096527 27901 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:57.096719 27901 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:57.135880 27901 rpc_server.cc:307] RPC server started. Bound to: 127.27.63.67:43321
I20260709 13:14:57.135994  6375 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.63.67:43321 every 8 connection(s)
I20260709 13:14:57.150025  6376 heartbeater.cc:344] Connected to a master server at 127.27.63.126:33459
I20260709 13:14:57.150503  6376 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:57.151178  6376 heartbeater.cc:507] Master 127.27.63.126:33459 requested a full tablet report, sending...
I20260709 13:14:57.153246  6100 ts_manager.cc:194] Registered new tserver with Master: b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:57.153899 27901 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014797652s
I20260709 13:14:57.154927  6100 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36334
I20260709 13:14:57.179831  6100 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36344:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:57.182367  6100 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:57.254374  6341 tablet_service.cc:1511] Processing CreateTablet for tablet 955fe372a4e74944ba40190825e7c20c (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:57.255929  6341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955fe372a4e74944ba40190825e7c20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.260198  6340 tablet_service.cc:1511] Processing CreateTablet for tablet 3b349dfc09f14259b1cce44dc222096f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:57.261631  6340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b349dfc09f14259b1cce44dc222096f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.263553  6267 tablet_service.cc:1511] Processing CreateTablet for tablet 955fe372a4e74944ba40190825e7c20c (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:57.265012  6267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955fe372a4e74944ba40190825e7c20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.264945  6265 tablet_service.cc:1511] Processing CreateTablet for tablet c41a951e84424e2dbc8f7994aeece051 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:57.266361  6265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c41a951e84424e2dbc8f7994aeece051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.268378  6339 tablet_service.cc:1511] Processing CreateTablet for tablet c41a951e84424e2dbc8f7994aeece051 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:57.269198  6266 tablet_service.cc:1511] Processing CreateTablet for tablet 3b349dfc09f14259b1cce44dc222096f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:57.269790  6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c41a951e84424e2dbc8f7994aeece051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.270507  6266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b349dfc09f14259b1cce44dc222096f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.278179  6264 tablet_service.cc:1511] Processing CreateTablet for tablet 6661734bc98940e1b967e62ae244ad39 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:57.278578  6338 tablet_service.cc:1511] Processing CreateTablet for tablet 6661734bc98940e1b967e62ae244ad39 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:57.279601  6264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6661734bc98940e1b967e62ae244ad39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.279973  6338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6661734bc98940e1b967e62ae244ad39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.283110  6192 tablet_service.cc:1511] Processing CreateTablet for tablet 955fe372a4e74944ba40190825e7c20c (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:57.284485  6192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955fe372a4e74944ba40190825e7c20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.289824  6191 tablet_service.cc:1511] Processing CreateTablet for tablet 3b349dfc09f14259b1cce44dc222096f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:57.291210  6191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b349dfc09f14259b1cce44dc222096f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.293701  6190 tablet_service.cc:1511] Processing CreateTablet for tablet c41a951e84424e2dbc8f7994aeece051 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:57.295117  6190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c41a951e84424e2dbc8f7994aeece051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.302055  6189 tablet_service.cc:1511] Processing CreateTablet for tablet 6661734bc98940e1b967e62ae244ad39 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:57.303498  6189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6661734bc98940e1b967e62ae244ad39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.303969  6263 tablet_service.cc:1511] Processing CreateTablet for tablet 38892ac1ee81491bb340bf1157637db2 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:57.305299  6263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38892ac1ee81491bb340bf1157637db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.317795  6188 tablet_service.cc:1511] Processing CreateTablet for tablet 38892ac1ee81491bb340bf1157637db2 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:57.319250  6188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38892ac1ee81491bb340bf1157637db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.324156  6337 tablet_service.cc:1511] Processing CreateTablet for tablet 38892ac1ee81491bb340bf1157637db2 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:57.325568  6337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38892ac1ee81491bb340bf1157637db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.336764  6187 tablet_service.cc:1511] Processing CreateTablet for tablet 89d628bd6ea04eeab2c7dc4a0da5b14f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:57.338689  6187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d628bd6ea04eeab2c7dc4a0da5b14f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.358523  6393 tablet_bootstrap.cc:492] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:57.374992  6393 tablet_bootstrap.cc:654] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.387434  6391 tablet_bootstrap.cc:492] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:14:57.415128  6264 tablet_service.cc:1511] Processing CreateTablet for tablet 89d628bd6ea04eeab2c7dc4a0da5b14f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:57.416640  6264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d628bd6ea04eeab2c7dc4a0da5b14f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.418673  6341 tablet_service.cc:1511] Processing CreateTablet for tablet 89d628bd6ea04eeab2c7dc4a0da5b14f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:57.426070  6341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d628bd6ea04eeab2c7dc4a0da5b14f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.427995  6392 tablet_bootstrap.cc:492] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.427908  6267 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba75b94277b4e21adc71d3343f6f0f2 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:57.429419  6267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba75b94277b4e21adc71d3343f6f0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.431464  6340 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba75b94277b4e21adc71d3343f6f0f2 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:57.432948  6340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba75b94277b4e21adc71d3343f6f0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.434580  6392 tablet_bootstrap.cc:654] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.443518  6391 tablet_bootstrap.cc:654] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.444300  6187 tablet_service.cc:1511] Processing CreateTablet for tablet 4456d13ebceb426687e25825f3ba949f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:57.450644  6187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4456d13ebceb426687e25825f3ba949f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.454629  6337 tablet_service.cc:1511] Processing CreateTablet for tablet 4456d13ebceb426687e25825f3ba949f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:57.455966  6337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4456d13ebceb426687e25825f3ba949f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.455756  6188 tablet_service.cc:1511] Processing CreateTablet for tablet d8171963e8d743b0a09a35917dc3474b (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:57.457597  6188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8171963e8d743b0a09a35917dc3474b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.460937  6392 tablet_bootstrap.cc:492] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:57.461504  6392 ts_tablet_manager.cc:1403] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.016s
I20260709 13:14:57.461560  6264 tablet_service.cc:1511] Processing CreateTablet for tablet 4456d13ebceb426687e25825f3ba949f (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:57.445497  6393 tablet_bootstrap.cc:492] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:14:57.462896  6264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4456d13ebceb426687e25825f3ba949f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.463101  6393 ts_tablet_manager.cc:1403] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.014s
I20260709 13:14:57.464773  6392 raft_consensus.cc:359] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.470533  6392 raft_consensus.cc:385] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.470906  6392 raft_consensus.cc:740] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.447031  6190 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba75b94277b4e21adc71d3343f6f0f2 (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:57.471709  6392 consensus_queue.cc:260] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.472852  6190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba75b94277b4e21adc71d3343f6f0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.473663  6391 tablet_bootstrap.cc:492] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:57.474234  6391 ts_tablet_manager.cc:1403] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.087s	user 0.010s	sys 0.003s
I20260709 13:14:57.477301  6391 raft_consensus.cc:359] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.479092  6391 raft_consensus.cc:385] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.479445  6391 raft_consensus.cc:740] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.480307  6391 consensus_queue.cc:260] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.488709  6393 raft_consensus.cc:359] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.493165  6393 raft_consensus.cc:385] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.493667  6393 raft_consensus.cc:740] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.494303  6267 tablet_service.cc:1511] Processing CreateTablet for tablet d8171963e8d743b0a09a35917dc3474b (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:57.496977  6391 ts_tablet_manager.cc:1434] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.007s
I20260709 13:14:57.498631  6376 heartbeater.cc:499] Master 127.27.63.126:33459 was elected leader, sending a full tablet report...
I20260709 13:14:57.500859  6267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8171963e8d743b0a09a35917dc3474b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.500754  6393 consensus_queue.cc:260] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.501271  6392 ts_tablet_manager.cc:1434] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.039s	user 0.013s	sys 0.020s
I20260709 13:14:57.501977  6302 heartbeater.cc:499] Master 127.27.63.126:33459 was elected leader, sending a full tablet report...
I20260709 13:14:57.502292  6392 tablet_bootstrap.cc:492] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.511696  6391 tablet_bootstrap.cc:492] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:14:57.517961  6391 tablet_bootstrap.cc:654] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.517956  6392 tablet_bootstrap.cc:654] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.542850  6337 tablet_service.cc:1511] Processing CreateTablet for tablet d8171963e8d743b0a09a35917dc3474b (DEFAULT_TABLE table=test-workload [id=319d41fd1af247fb8d97b5d718c0f2ee]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:57.543524  6227 heartbeater.cc:499] Master 127.27.63.126:33459 was elected leader, sending a full tablet report...
I20260709 13:14:57.544169  6337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8171963e8d743b0a09a35917dc3474b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:57.550639  6393 ts_tablet_manager.cc:1434] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.004s
I20260709 13:14:57.551798  6393 tablet_bootstrap.cc:492] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:57.557533  6392 tablet_bootstrap.cc:492] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:57.558758  6392 ts_tablet_manager.cc:1403] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.057s	user 0.027s	sys 0.019s
I20260709 13:14:57.559814  6391 tablet_bootstrap.cc:492] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:57.562290  6391 ts_tablet_manager.cc:1403] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.023s
I20260709 13:14:57.562304  6393 tablet_bootstrap.cc:654] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.562953  6392 raft_consensus.cc:359] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.563794  6392 raft_consensus.cc:385] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.564185  6392 raft_consensus.cc:740] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.565228  6392 consensus_queue.cc:260] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.565799  6391 raft_consensus.cc:359] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.567169  6391 raft_consensus.cc:385] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.567607  6391 raft_consensus.cc:740] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.568727  6391 consensus_queue.cc:260] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.570158  6392 ts_tablet_manager.cc:1434] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.002s
I20260709 13:14:57.570386  6393 tablet_bootstrap.cc:492] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:14:57.570978  6393 ts_tablet_manager.cc:1403] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.001s
I20260709 13:14:57.571230  6392 tablet_bootstrap.cc:492] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.572721  6391 ts_tablet_manager.cc:1434] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:14:57.573767  6391 tablet_bootstrap.cc:492] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:14:57.578209  6392 tablet_bootstrap.cc:654] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.580282  6393 raft_consensus.cc:359] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.581169  6393 raft_consensus.cc:385] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.581619  6393 raft_consensus.cc:740] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.582568  6391 tablet_bootstrap.cc:654] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.582832  6393 consensus_queue.cc:260] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.584828  6401 raft_consensus.cc:493] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.585420  6401 raft_consensus.cc:515] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.587935  6391 tablet_bootstrap.cc:492] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:57.588322  6392 tablet_bootstrap.cc:492] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:57.589125  6392 ts_tablet_manager.cc:1403] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.013s
I20260709 13:14:57.588572  6391 ts_tablet_manager.cc:1403] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.015s	user 0.004s	sys 0.008s
I20260709 13:14:57.593084  6391 raft_consensus.cc:359] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.594064  6391 raft_consensus.cc:385] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.593473  6392 raft_consensus.cc:359] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.594455  6391 raft_consensus.cc:740] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.594769  6392 raft_consensus.cc:385] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.589298  6401 leader_election.cc:290] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:57.595364  6392 raft_consensus.cc:740] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.596890  6392 consensus_queue.cc:260] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.598435  6391 consensus_queue.cc:260] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.599870  6392 ts_tablet_manager.cc:1434] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 13:14:57.600803  6391 ts_tablet_manager.cc:1434] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:14:57.600973  6392 tablet_bootstrap.cc:492] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.601827  6391 tablet_bootstrap.cc:492] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:14:57.607534  6392 tablet_bootstrap.cc:654] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.614943  6391 tablet_bootstrap.cc:654] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.616123  6392 tablet_bootstrap.cc:492] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:57.616760  6392 ts_tablet_manager.cc:1403] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.007s
I20260709 13:14:57.620422  6392 raft_consensus.cc:359] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.621515  6392 raft_consensus.cc:385] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.621985  6392 raft_consensus.cc:740] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.623202  6392 consensus_queue.cc:260] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.629077  6392 ts_tablet_manager.cc:1434] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:14:57.630609  6392 tablet_bootstrap.cc:492] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.633806  6391 tablet_bootstrap.cc:492] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:57.634475  6391 ts_tablet_manager.cc:1403] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.012s
I20260709 13:14:57.637952  6392 tablet_bootstrap.cc:654] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.637688  6391 raft_consensus.cc:359] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:57.638552  6391 raft_consensus.cc:385] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.638952  6391 raft_consensus.cc:740] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.641772  6391 consensus_queue.cc:260] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:57.649947  6391 ts_tablet_manager.cc:1434] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:14:57.653915  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6661734bc98940e1b967e62ae244ad39" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:57.654423  6391 tablet_bootstrap.cc:492] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:14:57.654991  6277 raft_consensus.cc:2468] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 0.
I20260709 13:14:57.659300  6167 leader_election.cc:304] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:57.662686  6401 raft_consensus.cc:2804] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:57.663175  6401 raft_consensus.cc:493] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.663569  6401 raft_consensus.cc:3060] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.674679  6391 tablet_bootstrap.cc:654] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.684306  6401 raft_consensus.cc:515] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.688239  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6661734bc98940e1b967e62ae244ad39" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:57.689019  6277 raft_consensus.cc:3060] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.708899  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6661734bc98940e1b967e62ae244ad39" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
W20260709 13:14:57.711067  6167 leader_election.cc:343] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:57.713984  6393 ts_tablet_manager.cc:1434] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.143s	user 0.051s	sys 0.070s
I20260709 13:14:57.715390  6392 tablet_bootstrap.cc:492] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:57.715938  6392 ts_tablet_manager.cc:1403] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.086s	user 0.034s	sys 0.046s
I20260709 13:14:57.715785  6277 raft_consensus.cc:2468] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 1.
I20260709 13:14:57.717913  6401 leader_election.cc:290] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:57.719010  6392 raft_consensus.cc:359] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:57.719781  6392 raft_consensus.cc:385] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.720106  6392 raft_consensus.cc:740] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.720896  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6661734bc98940e1b967e62ae244ad39" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:57.720808  6392 consensus_queue.cc:260] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:57.721418  6391 tablet_bootstrap.cc:492] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:57.720903  6167 leader_election.cc:304] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:57.722014  6391 ts_tablet_manager.cc:1403] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.068s	user 0.027s	sys 0.027s
I20260709 13:14:57.722589  6401 raft_consensus.cc:2804] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:57.723055  6401 raft_consensus.cc:697] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:14:57.723135  6392 ts_tablet_manager.cc:1434] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:14:57.724206  6392 tablet_bootstrap.cc:492] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.723891  6401 consensus_queue.cc:237] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.725251  6391 raft_consensus.cc:359] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.726091  6391 raft_consensus.cc:385] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.726481  6391 raft_consensus.cc:740] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.727274  6391 consensus_queue.cc:260] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
W20260709 13:14:57.729354  6167 leader_election.cc:343] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:57.729573  6391 ts_tablet_manager.cc:1434] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:57.721458  6393 tablet_bootstrap.cc:492] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:57.730768  6391 tablet_bootstrap.cc:492] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:14:57.731575  6392 tablet_bootstrap.cc:654] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.737748  6391 tablet_bootstrap.cc:654] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.742983  6393 tablet_bootstrap.cc:654] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.747990  6099 catalog_manager.cc:5697] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:57.750255  6398 raft_consensus.cc:493] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.750754  6398 raft_consensus.cc:515] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.756794  6398 leader_election.cc:290] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:57.758277  6392 tablet_bootstrap.cc:492] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:57.760555  6392 ts_tablet_manager.cc:1403] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.014s
I20260709 13:14:57.764410  6392 raft_consensus.cc:359] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.765489  6392 raft_consensus.cc:385] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.765864  6392 raft_consensus.cc:740] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.766724  6392 consensus_queue.cc:260] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
W20260709 13:14:57.769263  6148 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:57.767355  6391 tablet_bootstrap.cc:492] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:57.770037  6392 ts_tablet_manager.cc:1434] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.007s
I20260709 13:14:57.770318  6391 ts_tablet_manager.cc:1403] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.000s
I20260709 13:14:57.771359  6392 tablet_bootstrap.cc:492] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.773605  6391 raft_consensus.cc:359] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.776117  6391 raft_consensus.cc:385] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.776878  6391 raft_consensus.cc:740] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.778002  6391 consensus_queue.cc:260] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.780428  6392 tablet_bootstrap.cc:654] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.782409  6391 ts_tablet_manager.cc:1434] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:14:57.784797  6391 tablet_bootstrap.cc:492] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
W20260709 13:14:57.799841  6303 tablet.cc:2406] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:14:57.801704  6398 raft_consensus.cc:493] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.803175  6393 tablet_bootstrap.cc:492] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:14:57.802591  6398 raft_consensus.cc:515] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:57.803735  6393 ts_tablet_manager.cc:1403] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.024s
I20260709 13:14:57.805500  6391 tablet_bootstrap.cc:654] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.806828  6393 raft_consensus.cc:359] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:57.807624  6393 raft_consensus.cc:385] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.807906  6393 raft_consensus.cc:740] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.808830  6398 leader_election.cc:290] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369)
I20260709 13:14:57.808627  6393 consensus_queue.cc:260] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:57.811480  6392 tablet_bootstrap.cc:492] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:57.812213  6392 ts_tablet_manager.cc:1403] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.000s
I20260709 13:14:57.812240  6393 ts_tablet_manager.cc:1434] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:14:57.814550  6393 tablet_bootstrap.cc:492] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:57.816133  6392 raft_consensus.cc:359] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.816995  6392 raft_consensus.cc:385] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.817309  6392 raft_consensus.cc:740] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.818274  6392 consensus_queue.cc:260] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.821071  6392 ts_tablet_manager.cc:1434] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:14:57.822083  6392 tablet_bootstrap.cc:492] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.825635  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:57.826357  6277 raft_consensus.cc:2468] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:14:57.827497  6276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955fe372a4e74944ba40190825e7c20c" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:57.828470  6393 tablet_bootstrap.cc:654] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.828575  6276 raft_consensus.cc:2468] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:14:57.830106  6316 leader_election.cc:304] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:57.830966  6398 raft_consensus.cc:2804] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:57.831427  6398 raft_consensus.cc:493] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.831789  6398 raft_consensus.cc:3060] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.833638  6397 raft_consensus.cc:493] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.834332  6397 raft_consensus.cc:515] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.831092  6316 leader_election.cc:304] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:57.839268  6397 leader_election.cc:290] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:57.841535  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:14:57.846464  6391 tablet_bootstrap.cc:492] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:57.847085  6391 ts_tablet_manager.cc:1403] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.001s
I20260709 13:14:57.850534  6391 raft_consensus.cc:359] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.863023  6391 raft_consensus.cc:385] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.863420  6391 raft_consensus.cc:740] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.864442  6391 consensus_queue.cc:260] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.854251  6392 tablet_bootstrap.cc:654] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.861425  6393 tablet_bootstrap.cc:492] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
W20260709 13:14:57.883539  6314 leader_election.cc:343] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:57.884835  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955fe372a4e74944ba40190825e7c20c" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:14:57.885634  6201 raft_consensus.cc:2468] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:14:57.889287  6398 raft_consensus.cc:515] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.891414  6398 leader_election.cc:290] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:57.891968  6398 raft_consensus.cc:2804] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:57.874181  6393 ts_tablet_manager.cc:1403] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.000s
I20260709 13:14:57.895746  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:57.898283  6397 raft_consensus.cc:493] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:57.895917  6316 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.66:36991, user_credentials={real_user=slave}} blocked reactor thread for 64903.3us
W20260709 13:14:57.908288  6314 leader_election.cc:343] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:57.898904  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:57.908413  6397 raft_consensus.cc:515] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.896880  6398 raft_consensus.cc:493] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.910132  6351 raft_consensus.cc:2468] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:14:57.910318  6398 raft_consensus.cc:3060] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.896034  6426 raft_consensus.cc:493] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.909610  6276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:57.911482  6242 leader_election.cc:304] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:57.911948  6276 raft_consensus.cc:3060] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.913076  6397 leader_election.cc:290] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:57.913669  6397 raft_consensus.cc:2804] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:57.914067  6397 raft_consensus.cc:493] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.914453  6397 raft_consensus.cc:3060] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.915670  6401 raft_consensus.cc:493] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.916170  6401 raft_consensus.cc:515] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.915333  6391 ts_tablet_manager.cc:1434] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.068s	user 0.016s	sys 0.003s
I20260709 13:14:57.918213  6401 leader_election.cc:290] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:57.915416  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
W20260709 13:14:57.920018  6240 leader_election.cc:343] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:57.917676  6391 tablet_bootstrap.cc:492] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:14:57.920449  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:14:57.921422  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:57.921770  6202 raft_consensus.cc:2468] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:14:57.922103  6351 raft_consensus.cc:2468] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 0.
I20260709 13:14:57.923332  6167 leader_election.cc:304] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: 
I20260709 13:14:57.924221  6401 raft_consensus.cc:2804] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:57.923645  6426 raft_consensus.cc:515] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.924681  6401 raft_consensus.cc:493] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.925129  6401 raft_consensus.cc:3060] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.946805  6392 tablet_bootstrap.cc:492] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:57.947381  6392 ts_tablet_manager.cc:1403] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.126s	user 0.016s	sys 0.026s
I20260709 13:14:57.947335  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
W20260709 13:14:57.949824  6314 leader_election.cc:343] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:57.950330  6392 raft_consensus.cc:359] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.951125  6392 raft_consensus.cc:385] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.951382  6392 raft_consensus.cc:740] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.952018  6392 consensus_queue.cc:260] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.952464  6276 raft_consensus.cc:2468] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:14:57.955298  6426 leader_election.cc:290] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:57.957199  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:57.957465  6316 leader_election.cc:304] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:57.958155  6277 raft_consensus.cc:2468] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:14:57.958825  6426 raft_consensus.cc:2804] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:57.959687  6316 leader_election.cc:304] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:57.959362  6426 raft_consensus.cc:697] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:57.960690  6427 raft_consensus.cc:2804] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:57.957161  6429 raft_consensus.cc:493] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.961151  6427 raft_consensus.cc:493] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.960973  6426 consensus_queue.cc:237] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.961746  6427 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.961275  6429 raft_consensus.cc:515] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.963922  6392 ts_tablet_manager.cc:1434] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.008s
I20260709 13:14:57.964388  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
W20260709 13:14:57.965777  6240 leader_election.cc:343] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:57.966423  6393 raft_consensus.cc:359] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.972289  6393 raft_consensus.cc:385] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:57.967036  6429 leader_election.cc:290] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:57.972641  6393 raft_consensus.cc:740] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:57.967201  6392 tablet_bootstrap.cc:492] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:14:57.969645  6427 raft_consensus.cc:515] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.974822  6391 tablet_bootstrap.cc:654] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.975683  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
W20260709 13:14:57.977191  6314 leader_election.cc:343] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:57.970618  6242 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.63.67:43321, user_credentials={real_user=slave}} blocked reactor thread for 59321.4us
I20260709 13:14:57.971925  6401 raft_consensus.cc:515] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.978340  6397 raft_consensus.cc:515] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.976887  6398 raft_consensus.cc:515] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:57.979029  6393 consensus_queue.cc:260] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:57.979357  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:57.980223  6351 raft_consensus.cc:2393] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd18efee76df4c2699bff236b9d056ba in current term 1: Already voted for candidate b3ae2edef6a0467eaf647236a2c51703 in this term.
I20260709 13:14:57.981290  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:57.981390  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:57.982317  6202 raft_consensus.cc:2393] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dd18efee76df4c2699bff236b9d056ba in current term 1: Already voted for candidate 16719ad7c0bd4042acd734beca09a9e0 in this term.
I20260709 13:14:57.983989  6397 leader_election.cc:290] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:57.984728  6393 ts_tablet_manager.cc:1434] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.077s	user 0.002s	sys 0.005s
I20260709 13:14:57.985970  6276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:57.986126  6393 tablet_bootstrap.cc:492] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:57.987058  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:57.988768  6277 raft_consensus.cc:2393] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16719ad7c0bd4042acd734beca09a9e0 in current term 1: Already voted for candidate dd18efee76df4c2699bff236b9d056ba in this term.
I20260709 13:14:57.989130  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:57.989496  6276 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.989852  6350 raft_consensus.cc:3060] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.995574  6393 tablet_bootstrap.cc:654] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:57.996695  6242 leader_election.cc:304] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703
I20260709 13:14:57.997721  6350 raft_consensus.cc:2468] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
W20260709 13:14:57.997978  6242 leader_election.cc:343] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:57.998492  6242 leader_election.cc:304] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703
I20260709 13:14:57.998790  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955fe372a4e74944ba40190825e7c20c" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:57.999251  6429 raft_consensus.cc:2749] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:57.999523  6277 raft_consensus.cc:3060] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:57.988922  6427 leader_election.cc:290] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:57.999758  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955fe372a4e74944ba40190825e7c20c" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:58.000535  6202 raft_consensus.cc:3060] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.004835  6276 raft_consensus.cc:2468] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:14:58.005419  6397 raft_consensus.cc:2749] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:58.005893  6316 leader_election.cc:304] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.006711  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:58.007226  6427 raft_consensus.cc:2804] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:58.007598  6275 raft_consensus.cc:2393] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 16719ad7c0bd4042acd734beca09a9e0 in current term 1: Already voted for candidate dd18efee76df4c2699bff236b9d056ba in this term.
I20260709 13:14:58.007946  6427 raft_consensus.cc:697] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:58.008524  6242 leader_election.cc:304] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:57.987476  6392 tablet_bootstrap.cc:654] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:58.009807  6397 raft_consensus.cc:2804] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:58.010367  6397 raft_consensus.cc:697] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:14:58.011082  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.011202  6397 consensus_queue.cc:237] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.011938  6350 raft_consensus.cc:2393] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 16719ad7c0bd4042acd734beca09a9e0 in current term 1: Already voted for candidate dd18efee76df4c2699bff236b9d056ba in this term.
I20260709 13:14:58.011706  6398 leader_election.cc:290] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369)
I20260709 13:14:58.009003  6277 raft_consensus.cc:2468] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:14:58.012140  6401 leader_election.cc:290] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.014436  6316 leader_election.cc:304] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.014717  6202 raft_consensus.cc:2468] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:14:58.008903  6427 consensus_queue.cc:237] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.017856  6398 raft_consensus.cc:2804] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:58.018489  6167 leader_election.cc:304] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0; no voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba
I20260709 13:14:58.020022  6401 raft_consensus.cc:2749] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:58.020826  6398 raft_consensus.cc:697] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:58.022840  6392 tablet_bootstrap.cc:492] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:14:58.023604  6392 ts_tablet_manager.cc:1403] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.001s
I20260709 13:14:58.023517  6398 consensus_queue.cc:237] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:58.030385  6100 catalog_manager.cc:5697] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:58.027383  6392 raft_consensus.cc:359] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.031836  6392 raft_consensus.cc:385] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:58.032197  6392 raft_consensus.cc:740] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:14:58.032023  6099 catalog_manager.cc:5697] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 0 to 1, leader changed from <none> to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 1 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.033013  6392 consensus_queue.cc:260] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.034185  6393 tablet_bootstrap.cc:492] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:14:58.034789  6393 ts_tablet_manager.cc:1403] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.049s	user 0.007s	sys 0.005s
I20260709 13:14:58.035012  6392 ts_tablet_manager.cc:1434] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:14:58.037132  6391 tablet_bootstrap.cc:492] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:58.037824  6391 ts_tablet_manager.cc:1403] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.120s	user 0.017s	sys 0.020s
I20260709 13:14:58.037957  6393 raft_consensus.cc:359] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.038892  6393 raft_consensus.cc:385] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:58.039227  6393 raft_consensus.cc:740] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:58.040174  6393 consensus_queue.cc:260] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.041492  6391 raft_consensus.cc:359] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.042479  6391 raft_consensus.cc:385] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:58.042807  6391 raft_consensus.cc:740] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:58.043701  6391 consensus_queue.cc:260] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.046504  6391 ts_tablet_manager.cc:1434] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:14:58.047837  6391 tablet_bootstrap.cc:492] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:14:58.046518  6393 ts_tablet_manager.cc:1434] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:58.054828  6393 tablet_bootstrap.cc:492] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:58.055037  6391 tablet_bootstrap.cc:654] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:58.064877  6391 tablet_bootstrap.cc:492] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:14:58.065356  6391 ts_tablet_manager.cc:1403] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.001s
I20260709 13:14:58.065551  6393 tablet_bootstrap.cc:654] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:58.068466  6391 raft_consensus.cc:359] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.068871  6099 catalog_manager.cc:5697] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:58.070153  6391 raft_consensus.cc:385] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:58.070581  6397 raft_consensus.cc:493] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.071144  6397 raft_consensus.cc:515] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.070632  6391 raft_consensus.cc:740] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:14:58.073314  6397 leader_election.cc:290] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.071144  6099 catalog_manager.cc:5697] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.073189  6391 consensus_queue.cc:260] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.075235  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:14:58.076180  6391 ts_tablet_manager.cc:1434] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:14:58.076476  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:58.076750  6393 tablet_bootstrap.cc:492] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:14:58.077137  6350 raft_consensus.cc:2468] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:14:58.077325  6393 ts_tablet_manager.cc:1403] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.006s
W20260709 13:14:58.077844  6240 leader_election.cc:343] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:58.078500  6398 raft_consensus.cc:493] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.078511  6242 leader_election.cc:304] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.079162  6398 raft_consensus.cc:515] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.079908  6397 raft_consensus.cc:2804] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:58.080291  6397 raft_consensus.cc:493] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.080622  6397 raft_consensus.cc:3060] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.080624  6393 raft_consensus.cc:359] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.081344  6398 leader_election.cc:290] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:58.081573  6393 raft_consensus.cc:385] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:58.081898  6393 raft_consensus.cc:740] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:58.082636  6393 consensus_queue.cc:260] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.083000  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:14:58.083993  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:58.084273  6401 raft_consensus.cc:493] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.084678  6277 raft_consensus.cc:2468] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:14:58.084707  6401 raft_consensus.cc:515] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.086037  6316 leader_election.cc:304] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.086822  6398 raft_consensus.cc:2804] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:58.086856  6401 leader_election.cc:290] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.087329  6398 raft_consensus.cc:493] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.086901  6393 ts_tablet_manager.cc:1434] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:14:58.087800  6398 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:58.087965  6314 leader_election.cc:343] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:58.088670  6393 tablet_bootstrap.cc:492] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:58.089306  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:58.089603  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:58.090195  6277 raft_consensus.cc:2393] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16719ad7c0bd4042acd734beca09a9e0 in current term 1: Already voted for candidate b3ae2edef6a0467eaf647236a2c51703 in this term.
I20260709 13:14:58.091414  6167 leader_election.cc:304] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0; no voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba
I20260709 13:14:58.092208  6401 raft_consensus.cc:3060] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.096375  6398 raft_consensus.cc:515] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.097420  6401 raft_consensus.cc:2749] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:58.099761  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:58.100536  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:58.101146  6393 tablet_bootstrap.cc:654] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:58.101197  6277 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:58.101509  6314 leader_election.cc:343] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:58.105887  6429 raft_consensus.cc:493] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.106588  6429 raft_consensus.cc:515] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.103451  6397 raft_consensus.cc:515] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.109494  6397 leader_election.cc:290] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.111503  6429 leader_election.cc:290] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.111884  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.112566  6351 raft_consensus.cc:3060] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.113260  6393 tablet_bootstrap.cc:492] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:14:58.113853  6393 ts_tablet_manager.cc:1403] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.006s
I20260709 13:14:58.113899  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:58.111537  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89d628bd6ea04eeab2c7dc4a0da5b14f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:58.115615  6350 raft_consensus.cc:2468] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
W20260709 13:14:58.116933  6240 leader_election.cc:343] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:58.117409  6242 leader_election.cc:304] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.117192  6393 raft_consensus.cc:359] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.117863  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89d628bd6ea04eeab2c7dc4a0da5b14f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:14:58.118110  6393 raft_consensus.cc:385] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:58.118527  6429 raft_consensus.cc:2804] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:58.118568  6202 raft_consensus.cc:2468] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:14:58.119017  6429 raft_consensus.cc:493] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.119428  6429 raft_consensus.cc:3060] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.117873  6277 raft_consensus.cc:2468] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:14:58.118587  6398 leader_election.cc:290] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:58.118546  6393 raft_consensus.cc:740] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:58.121681  6316 leader_election.cc:304] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.121742  6393 consensus_queue.cc:260] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.122902  6351 raft_consensus.cc:2468] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:14:58.123202  6398 raft_consensus.cc:2804] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:58.123893  6398 raft_consensus.cc:697] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:58.124104  6242 leader_election.cc:304] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.124444  6393 ts_tablet_manager.cc:1434] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.003s
I20260709 13:14:58.125468  6397 raft_consensus.cc:2804] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:58.124980  6398 consensus_queue.cc:237] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.126014  6397 raft_consensus.cc:697] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:14:58.127242  6393 tablet_bootstrap.cc:492] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:58.126883  6397 consensus_queue.cc:237] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.132200  6429 raft_consensus.cc:515] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.134912  6429 leader_election.cc:290] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.135697  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89d628bd6ea04eeab2c7dc4a0da5b14f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:58.136409  6202 raft_consensus.cc:3060] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.136567  6393 tablet_bootstrap.cc:654] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:58.136704  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89d628bd6ea04eeab2c7dc4a0da5b14f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.137393  6351 raft_consensus.cc:3060] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.143625  6202 raft_consensus.cc:2468] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:14:58.145521  6240 leader_election.cc:304] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.145810  6393 tablet_bootstrap.cc:492] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:14:58.146390  6393 ts_tablet_manager.cc:1403] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.005s
I20260709 13:14:58.147442  6429 raft_consensus.cc:2804] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:58.148563  6429 raft_consensus.cc:697] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:14:58.149125  6099 catalog_manager.cc:5697] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:58.149797  6429 consensus_queue.cc:237] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.147531  6351 raft_consensus.cc:2468] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:14:58.152447  6393 raft_consensus.cc:359] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.153139  6393 raft_consensus.cc:385] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:58.153378  6393 raft_consensus.cc:740] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:58.156288  6393 consensus_queue.cc:260] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.159116  6397 raft_consensus.cc:493] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.159646  6397 raft_consensus.cc:515] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.161751  6397 leader_election.cc:290] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.162174  6393 ts_tablet_manager.cc:1434] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:14:58.161558  6097 catalog_manager.cc:5697] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 0 to 1, leader changed from <none> to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 1 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.163029  6393 tablet_bootstrap.cc:492] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:14:58.164079  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4456d13ebceb426687e25825f3ba949f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:14:58.164752  6202 raft_consensus.cc:2468] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:14:58.165899  6240 leader_election.cc:304] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.166700  6397 raft_consensus.cc:2804] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:58.167040  6397 raft_consensus.cc:493] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.167534  6397 raft_consensus.cc:3060] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.167992  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4456d13ebceb426687e25825f3ba949f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:58.168255  6393 tablet_bootstrap.cc:654] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:58.168711  6351 raft_consensus.cc:2468] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:14:58.170854  6398 raft_consensus.cc:493] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.171504  6398 raft_consensus.cc:515] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.174783  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6661734bc98940e1b967e62ae244ad39" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:14:58.175590  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6661734bc98940e1b967e62ae244ad39" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:58.176688  6277 raft_consensus.cc:2393] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3ae2edef6a0467eaf647236a2c51703 in current term 1: Already voted for candidate 16719ad7c0bd4042acd734beca09a9e0 in this term.
I20260709 13:14:58.176528  6097 catalog_manager.cc:5697] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 0 to 1, leader changed from <none> to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 1 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.178071  6316 leader_election.cc:304] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703; no voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba
I20260709 13:14:58.178424  6398 leader_election.cc:290] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:58.178853  6427 raft_consensus.cc:3060] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.181313  6393 tablet_bootstrap.cc:492] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:14:58.184149  6397 raft_consensus.cc:515] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.186277  6397 leader_election.cc:290] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.187865  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4456d13ebceb426687e25825f3ba949f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:58.188176  6393 ts_tablet_manager.cc:1403] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.012s
I20260709 13:14:58.188534  6202 raft_consensus.cc:3060] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.189792  6427 raft_consensus.cc:2749] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:58.190954  6393 raft_consensus.cc:359] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.192083  6393 raft_consensus.cc:385] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:58.192407  6393 raft_consensus.cc:740] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:58.192512  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4456d13ebceb426687e25825f3ba949f" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.193105  6350 raft_consensus.cc:3060] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.193161  6393 consensus_queue.cc:260] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.194867  6406 consensus_queue.cc:1048] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.195276  6393 ts_tablet_manager.cc:1434] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:58.196710  6202 raft_consensus.cc:2468] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:14:58.198100  6240 leader_election.cc:304] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.198944  6397 raft_consensus.cc:2804] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:58.200111  6397 raft_consensus.cc:697] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:14:58.200999  6397 consensus_queue.cc:237] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.202039  6350 raft_consensus.cc:2468] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:14:58.213023  6097 catalog_manager.cc:5697] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 0 to 1, leader changed from <none> to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 1 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.233862  6452 consensus_queue.cc:1048] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:58.331070  6452 raft_consensus.cc:493] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.331509  6452 raft_consensus.cc:515] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.333339  6452 leader_election.cc:290] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.334203  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:58.334447  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:58.335170  6351 raft_consensus.cc:2393] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16719ad7c0bd4042acd734beca09a9e0 in current term 1: Already voted for candidate dd18efee76df4c2699bff236b9d056ba in this term.
I20260709 13:14:58.336199  6167 leader_election.cc:304] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0; no voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba
I20260709 13:14:58.336884  6452 raft_consensus.cc:3060] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.337384  6401 raft_consensus.cc:493] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.337973  6401 raft_consensus.cc:515] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.340868  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:58.340920  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:58.341780  6277 raft_consensus.cc:2393] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16719ad7c0bd4042acd734beca09a9e0 in current term 1: Already voted for candidate b3ae2edef6a0467eaf647236a2c51703 in this term.
I20260709 13:14:58.342734  6401 leader_election.cc:290] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.343451  6167 leader_election.cc:304] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0; no voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba
I20260709 13:14:58.344697  6452 raft_consensus.cc:2749] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:58.345384  6452 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.349968  6452 raft_consensus.cc:2749] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:58.354061  6351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955fe372a4e74944ba40190825e7c20c"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
mode: GRACEFUL
new_leader_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:45564
I20260709 13:14:58.354673  6351 raft_consensus.cc:606] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Received request to transfer leadership to TS 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.368458  6277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
mode: GRACEFUL
new_leader_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:42888
I20260709 13:14:58.369180  6277 raft_consensus.cc:606] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Received request to transfer leadership to TS 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.375552  6351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
mode: GRACEFUL
new_leader_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:45564
I20260709 13:14:58.376160  6351 raft_consensus.cc:606] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Received request to transfer leadership to TS 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.384217  6351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
mode: GRACEFUL
new_leader_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:45564
I20260709 13:14:58.384738  6351 raft_consensus.cc:606] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Received request to transfer leadership to TS dd18efee76df4c2699bff236b9d056ba
I20260709 13:14:58.392194  6351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
mode: GRACEFUL
new_leader_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:45564
I20260709 13:14:58.392813  6351 raft_consensus.cc:606] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Received request to transfer leadership to TS dd18efee76df4c2699bff236b9d056ba
I20260709 13:14:58.400537  6277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
mode: GRACEFUL
new_leader_uuid: "b3ae2edef6a0467eaf647236a2c51703"
 from {username='slave'} at 127.0.0.1:42888
I20260709 13:14:58.401052  6277 raft_consensus.cc:606] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Received request to transfer leadership to TS b3ae2edef6a0467eaf647236a2c51703
I20260709 13:14:58.407605  6427 consensus_queue.cc:1048] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.417613  6426 consensus_queue.cc:1048] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.426872  6397 consensus_queue.cc:1048] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.427040  6452 raft_consensus.cc:493] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.427625  6452 raft_consensus.cc:515] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.428097  6398 raft_consensus.cc:993] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703: : Instructing follower 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:14:58.428651  6426 raft_consensus.cc:1081] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Signalling peer 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:14:58.429648  6452 leader_election.cc:290] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.430871  6202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:53102
I20260709 13:14:58.431485  6202 raft_consensus.cc:493] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.431865  6202 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.430996  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:14:58.433163  6277 raft_consensus.cc:2393] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16719ad7c0bd4042acd734beca09a9e0 in current term 1: Already voted for candidate b3ae2edef6a0467eaf647236a2c51703 in this term.
I20260709 13:14:58.433547  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:14:58.435029  6167 leader_election.cc:304] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0; no voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba
I20260709 13:14:58.437487  6452 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:58.440629  6202 raft_consensus.cc:515] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.442375  6430 consensus_queue.cc:1048] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.445762  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:58.446353  6277 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.449524  6426 consensus_queue.cc:1048] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.452643  6202 leader_election.cc:290] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.453822  6277 raft_consensus.cc:2468] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:14:58.455159  6167 leader_election.cc:304] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.455813  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.456090  6401 raft_consensus.cc:2804] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:58.456679  6401 raft_consensus.cc:697] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:14:58.457535  6401 consensus_queue.cc:237] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.458820  6430 raft_consensus.cc:993] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba: : Instructing follower 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:14:58.459262  6397 raft_consensus.cc:1081] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Signalling peer 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:14:58.466454  6201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:53104
I20260709 13:14:58.467077  6201 raft_consensus.cc:493] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.467466  6201 raft_consensus.cc:3060] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.474112  6426 consensus_queue.cc:1048] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.475083  6201 raft_consensus.cc:515] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.456383  6351 raft_consensus.cc:3055] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:58.480324  6351 raft_consensus.cc:740] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:58.481213  6351 consensus_queue.cc:260] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.482795  6351 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.485004  6097 catalog_manager.cc:5697] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 1 to 2, leader changed from b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67) to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.487219  6426 raft_consensus.cc:993] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703: : Instructing follower 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:14:58.487430  6452 raft_consensus.cc:2749] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:58.487756  6426 raft_consensus.cc:1081] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Signalling peer 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:14:58.489334  6398 consensus_queue.cc:1048] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.490265  6202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "955fe372a4e74944ba40190825e7c20c"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:53102
I20260709 13:14:58.490876  6202 raft_consensus.cc:493] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.491230  6202 raft_consensus.cc:3060] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.492923  6351 raft_consensus.cc:2468] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:14:58.493767  6276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:58.494342  6276 raft_consensus.cc:3055] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:58.494659  6276 raft_consensus.cc:740] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:14:58.495520  6276 consensus_queue.cc:260] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.496884  6276 raft_consensus.cc:3060] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.498571  6202 raft_consensus.cc:515] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:58.500744  6202 leader_election.cc:290] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Requested vote from peers b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:58.501869  6201 leader_election.cc:290] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.502882  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955fe372a4e74944ba40190825e7c20c" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.503556  6351 raft_consensus.cc:3055] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:58.503866  6351 raft_consensus.cc:740] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:58.504207  6276 raft_consensus.cc:2468] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:14:58.504684  6351 consensus_queue.cc:260] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:58.505816  6167 leader_election.cc:304] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.506045  6351 raft_consensus.cc:3060] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.507330  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b349dfc09f14259b1cce44dc222096f" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.507930  6350 raft_consensus.cc:3060] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.511618  6351 raft_consensus.cc:2468] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:14:58.512755  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955fe372a4e74944ba40190825e7c20c" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:58.512851  6167 leader_election.cc:304] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: 
I20260709 13:14:58.513257  6427 raft_consensus.cc:993] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703: : Instructing follower dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:14:58.513707  6427 raft_consensus.cc:1081] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Signalling peer dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:14:58.513809  6401 raft_consensus.cc:2804] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:58.514290  6350 raft_consensus.cc:2468] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:14:58.514420  6401 raft_consensus.cc:697] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:14:58.515604  6401 consensus_queue.cc:237] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } }
I20260709 13:14:58.516229  6275 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:42874
I20260709 13:14:58.516815  6275 raft_consensus.cc:493] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.517166  6275 raft_consensus.cc:3060] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.513504  6277 raft_consensus.cc:3060] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.521665  6424 consensus_queue.cc:1048] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:58.512822  6452 raft_consensus.cc:2804] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:58.522936  6452 raft_consensus.cc:697] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:14:58.523772  6452 consensus_queue.cc:237] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.524827  6275 raft_consensus.cc:515] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.527217  6275 leader_election.cc:290] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.529080  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.529758  6350 raft_consensus.cc:3055] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:58.530117  6350 raft_consensus.cc:740] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:58.530977  6350 consensus_queue.cc:260] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.531418  6277 raft_consensus.cc:2468] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:14:58.532083  6350 raft_consensus.cc:3060] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.536392  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38892ac1ee81491bb340bf1157637db2" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:58.532583  6276 raft_consensus.cc:1240] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Rejecting Update request from peer b3ae2edef6a0467eaf647236a2c51703 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:58.542430  6097 catalog_manager.cc:5697] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 1 to 2, leader changed from b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67) to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:58.543887  6097 catalog_manager.cc:5697] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 1 to 2, leader changed from dd18efee76df4c2699bff236b9d056ba (127.27.63.66) to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.544584  6350 raft_consensus.cc:2468] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:14:58.545759  6242 leader_election.cc:304] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.546644  6429 raft_consensus.cc:2804] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:58.547158  6429 raft_consensus.cc:697] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:14:58.548022  6429 consensus_queue.cc:237] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.549609  6397 consensus_queue.cc:1048] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.557128  6099 catalog_manager.cc:5697] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 1 to 2, leader changed from b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67) to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 2 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.559019  6397 consensus_queue.cc:1048] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.567230  6430 consensus_queue.cc:1048] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.571344  6424 consensus_queue.cc:1048] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.584832  6397 raft_consensus.cc:993] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba: : Instructing follower b3ae2edef6a0467eaf647236a2c51703 to start an election
I20260709 13:14:58.585294  6397 raft_consensus.cc:1081] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Signalling peer b3ae2edef6a0467eaf647236a2c51703 to start an election
I20260709 13:14:58.586166  6424 consensus_queue.cc:1048] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.589037  6351 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
 from {username='slave'} at 127.0.0.1:45550
I20260709 13:14:58.589665  6351 raft_consensus.cc:493] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.590037  6351 raft_consensus.cc:3060] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.592921  6397 consensus_queue.cc:1048] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.602599  6351 raft_consensus.cc:515] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.603734  6350 raft_consensus.cc:1240] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Rejecting Update request from peer dd18efee76df4c2699bff236b9d056ba for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:58.606092  6397 consensus_queue.cc:1059] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.606979  6430 raft_consensus.cc:3055] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:58.607301  6430 raft_consensus.cc:740] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:14:58.608155  6430 consensus_queue.cc:260] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.608911  6426 raft_consensus.cc:993] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703: : Instructing follower dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:14:58.609086  6430 raft_consensus.cc:3060] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.609503  6426 raft_consensus.cc:1081] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Signalling peer dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:14:58.609633  6351 leader_election.cc:290] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 2 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:14:58.610708  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:14:58.611444  6275 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:42874
I20260709 13:14:58.612020  6275 raft_consensus.cc:493] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.612397  6275 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.615310  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8171963e8d743b0a09a35917dc3474b" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:58.620592  6275 raft_consensus.cc:515] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.625428  6275 leader_election.cc:290] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:14:58.626436  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:14:58.627142  6351 raft_consensus.cc:3055] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:58.627601  6277 raft_consensus.cc:2468] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 2.
I20260709 13:14:58.627526  6351 raft_consensus.cc:740] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:58.628159  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:14:58.628931  6201 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.628862  6351 consensus_queue.cc:260] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.630290  6351 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.630693  6316 leader_election.cc:304] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:14:58.631951  6427 raft_consensus.cc:2804] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:58.632465  6427 raft_consensus.cc:697] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 2 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:14:58.633282  6427 consensus_queue.cc:237] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.636550  6351 raft_consensus.cc:2468] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:14:58.636803  6201 raft_consensus.cc:2468] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:14:58.637866  6242 leader_election.cc:304] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:14:58.638672  6430 raft_consensus.cc:2804] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:58.639091  6430 raft_consensus.cc:697] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:14:58.639804  6430 consensus_queue.cc:237] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:14:58.641301  6097 catalog_manager.cc:5697] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 1 to 2, leader changed from dd18efee76df4c2699bff236b9d056ba (127.27.63.66) to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:58.647008  6099 catalog_manager.cc:5697] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 1 to 2, leader changed from b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67) to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 2 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.715035  6430 consensus_queue.cc:1048] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:58.725754  6430 consensus_queue.cc:1048] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:58.908555  6277 raft_consensus.cc:1275] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.910228  6452 consensus_queue.cc:1048] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.920980  6351 raft_consensus.cc:1275] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.925277  6401 consensus_queue.cc:1048] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.939487  6277 raft_consensus.cc:1275] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.940688  6401 consensus_queue.cc:1048] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.947583  6351 raft_consensus.cc:1275] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.950222  6406 consensus_queue.cc:1048] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.968724  6351 raft_consensus.cc:1275] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.970013  6406 consensus_queue.cc:1048] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.977429  6277 raft_consensus.cc:1275] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.979202  6406 consensus_queue.cc:1048] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:59.017819  6201 raft_consensus.cc:3060] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:59.023002  6201 raft_consensus.cc:1275] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Refusing update from remote peer b3ae2edef6a0467eaf647236a2c51703: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:59.024272  6427 consensus_queue.cc:1048] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:59.032166  6277 raft_consensus.cc:1275] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Refusing update from remote peer b3ae2edef6a0467eaf647236a2c51703: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:59.034816  6427 consensus_queue.cc:1048] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:59.039873  6201 raft_consensus.cc:3060] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:59.046641  6201 raft_consensus.cc:1275] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:59.047798  6430 consensus_queue.cc:1048] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:59.055147  6351 raft_consensus.cc:1275] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:59.056908  6430 consensus_queue.cc:1048] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:59.060667  6350 raft_consensus.cc:1275] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:59.064607  6429 consensus_queue.cc:1048] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:59.072201  6201 raft_consensus.cc:1275] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:59.073935  6430 consensus_queue.cc:1048] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.434376  6099 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36346:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:01.438165  6099 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:15:01.486701  6190 tablet_service.cc:1511] Processing CreateTablet for tablet 113550147ed34c9ea2d337c2aca91f4e (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:01.487807  6267 tablet_service.cc:1511] Processing CreateTablet for tablet 113550147ed34c9ea2d337c2aca91f4e (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:01.489173  6267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 113550147ed34c9ea2d337c2aca91f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.488180  6190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 113550147ed34c9ea2d337c2aca91f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.495343  6337 tablet_service.cc:1511] Processing CreateTablet for tablet 113550147ed34c9ea2d337c2aca91f4e (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:01.496822  6337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 113550147ed34c9ea2d337c2aca91f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.496871  6188 tablet_service.cc:1511] Processing CreateTablet for tablet 99808726cdea4e4897076087e79c92f4 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:01.498332  6188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99808726cdea4e4897076087e79c92f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.505682  6340 tablet_service.cc:1511] Processing CreateTablet for tablet 99808726cdea4e4897076087e79c92f4 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:01.507184  6340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99808726cdea4e4897076087e79c92f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.516629  6264 tablet_service.cc:1511] Processing CreateTablet for tablet 99808726cdea4e4897076087e79c92f4 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:01.517254  6266 tablet_service.cc:1511] Processing CreateTablet for tablet 8a10248efa3245f3afbde0526ce1d96d (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:01.518160  6264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99808726cdea4e4897076087e79c92f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.519088  6266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a10248efa3245f3afbde0526ce1d96d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.528899  6338 tablet_service.cc:1511] Processing CreateTablet for tablet 8a10248efa3245f3afbde0526ce1d96d (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:01.529359  6341 tablet_service.cc:1511] Processing CreateTablet for tablet 4299aff2799e405d8a924eb8227e6fdb (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:01.529613  6339 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6a8dd7983645259cbb2cc1f09c6e7c (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:01.530335  6338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a10248efa3245f3afbde0526ce1d96d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.530982  6341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4299aff2799e405d8a924eb8227e6fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.531280  6263 tablet_service.cc:1511] Processing CreateTablet for tablet 4299aff2799e405d8a924eb8227e6fdb (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:01.532716  6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6a8dd7983645259cbb2cc1f09c6e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.536691  6263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4299aff2799e405d8a924eb8227e6fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.541317  6265 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6a8dd7983645259cbb2cc1f09c6e7c (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:01.542840  6265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6a8dd7983645259cbb2cc1f09c6e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.545764  6262 tablet_service.cc:1511] Processing CreateTablet for tablet 4f8b5015e29d44d4a5cbe2d00f415a18 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:01.547302  6262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f8b5015e29d44d4a5cbe2d00f415a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.554189  6187 tablet_service.cc:1511] Processing CreateTablet for tablet 8a10248efa3245f3afbde0526ce1d96d (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:01.554845  6192 tablet_service.cc:1511] Processing CreateTablet for tablet 4299aff2799e405d8a924eb8227e6fdb (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:01.555753  6187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a10248efa3245f3afbde0526ce1d96d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.556960  6189 tablet_service.cc:1511] Processing CreateTablet for tablet 4f8b5015e29d44d4a5cbe2d00f415a18 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:01.557139  6191 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6a8dd7983645259cbb2cc1f09c6e7c (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:01.560375  6189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f8b5015e29d44d4a5cbe2d00f415a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.562491  6192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4299aff2799e405d8a924eb8227e6fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.564486  6191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6a8dd7983645259cbb2cc1f09c6e7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.568629  6186 tablet_service.cc:1511] Processing CreateTablet for tablet 875130b8f7db4b0faec1c151bc3cf694 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:01.569170  6336 tablet_service.cc:1511] Processing CreateTablet for tablet 4f8b5015e29d44d4a5cbe2d00f415a18 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:01.570230  6186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 875130b8f7db4b0faec1c151bc3cf694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.570564  6336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f8b5015e29d44d4a5cbe2d00f415a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.579047  6335 tablet_service.cc:1511] Processing CreateTablet for tablet 875130b8f7db4b0faec1c151bc3cf694 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:01.580560  6335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 875130b8f7db4b0faec1c151bc3cf694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.580613  6261 tablet_service.cc:1511] Processing CreateTablet for tablet 875130b8f7db4b0faec1c151bc3cf694 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:01.582237  6261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 875130b8f7db4b0faec1c151bc3cf694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.585451  6185 tablet_service.cc:1511] Processing CreateTablet for tablet 36330902a8014b269c0ed6ffaf54c238 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:01.586983  6185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36330902a8014b269c0ed6ffaf54c238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.589253  6260 tablet_service.cc:1511] Processing CreateTablet for tablet 36330902a8014b269c0ed6ffaf54c238 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:01.590770  6260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36330902a8014b269c0ed6ffaf54c238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.591213  6334 tablet_service.cc:1511] Processing CreateTablet for tablet 36330902a8014b269c0ed6ffaf54c238 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:01.592603  6334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36330902a8014b269c0ed6ffaf54c238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.606560  6259 tablet_service.cc:1511] Processing CreateTablet for tablet 824e6d3a7170481b9c540a57c7768843 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:01.613653  6486 tablet_bootstrap.cc:492] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:01.620671  6259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824e6d3a7170481b9c540a57c7768843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.642663  6339 tablet_service.cc:1511] Processing CreateTablet for tablet 824e6d3a7170481b9c540a57c7768843 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:01.677018  6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824e6d3a7170481b9c540a57c7768843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.686915  6484 tablet_bootstrap.cc:492] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:01.717121  6190 tablet_service.cc:1511] Processing CreateTablet for tablet 824e6d3a7170481b9c540a57c7768843 (DEFAULT_TABLE table=soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:01.731681  6486 tablet_bootstrap.cc:654] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.735819  6190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824e6d3a7170481b9c540a57c7768843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.743484  6484 tablet_bootstrap.cc:654] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.752832  6487 tablet_bootstrap.cc:492] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:15:01.779448  6487 tablet_bootstrap.cc:654] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.795202  6486 tablet_bootstrap.cc:492] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:15:01.795718  6487 tablet_bootstrap.cc:492] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:01.795725  6486 ts_tablet_manager.cc:1403] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.182s	user 0.007s	sys 0.008s
I20260709 13:15:01.796245  6484 tablet_bootstrap.cc:492] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:01.796361  6487 ts_tablet_manager.cc:1403] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.044s	user 0.002s	sys 0.016s
I20260709 13:15:01.796824  6484 ts_tablet_manager.cc:1403] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.006s
I20260709 13:15:01.799408  6487 raft_consensus.cc:359] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.799279  6486 raft_consensus.cc:359] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.800170  6487 raft_consensus.cc:385] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.799829  6484 raft_consensus.cc:359] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.800241  6486 raft_consensus.cc:385] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.800491  6487 raft_consensus.cc:740] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.800669  6484 raft_consensus.cc:385] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.800693  6486 raft_consensus.cc:740] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.801318  6484 raft_consensus.cc:740] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.801935  6486 consensus_queue.cc:260] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.802217  6484 consensus_queue.cc:260] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.801963  6487 consensus_queue.cc:260] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.804701  6484 ts_tablet_manager.cc:1434] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:01.805271  6487 ts_tablet_manager.cc:1434] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:01.805765  6484 tablet_bootstrap.cc:492] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:01.806502  6487 tablet_bootstrap.cc:492] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
W20260709 13:15:01.808749  6148 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:01.812846  6484 tablet_bootstrap.cc:654] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.815366  6487 tablet_bootstrap.cc:654] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.818786  6486 ts_tablet_manager.cc:1434] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 13:15:01.819530  6484 tablet_bootstrap.cc:492] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:01.819993  6486 tablet_bootstrap.cc:492] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:01.820135  6484 ts_tablet_manager.cc:1403] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.002s
I20260709 13:15:01.823484  6484 raft_consensus.cc:359] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.824276  6484 raft_consensus.cc:385] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.824568  6484 raft_consensus.cc:740] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.825569  6484 consensus_queue.cc:260] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.827028  6486 tablet_bootstrap.cc:654] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.828387  6484 ts_tablet_manager.cc:1434] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:01.829727  6484 tablet_bootstrap.cc:492] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:01.837414  6484 tablet_bootstrap.cc:654] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.842159  6487 tablet_bootstrap.cc:492] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:01.843010  6487 ts_tablet_manager.cc:1403] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.009s
I20260709 13:15:01.847129  6487 raft_consensus.cc:359] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.849190  6487 raft_consensus.cc:385] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.849542  6487 raft_consensus.cc:740] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.850358  6487 consensus_queue.cc:260] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.851153  6486 tablet_bootstrap.cc:492] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:15:01.851761  6486 ts_tablet_manager.cc:1403] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.016s
I20260709 13:15:01.851789  6484 tablet_bootstrap.cc:492] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:01.852403  6484 ts_tablet_manager.cc:1403] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.004s
I20260709 13:15:01.852778  6487 ts_tablet_manager.cc:1434] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:15:01.854357  6487 tablet_bootstrap.cc:492] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:15:01.855031  6486 raft_consensus.cc:359] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.855289  6484 raft_consensus.cc:359] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.855801  6486 raft_consensus.cc:385] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.856114  6484 raft_consensus.cc:385] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.856164  6486 raft_consensus.cc:740] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.856590  6484 raft_consensus.cc:740] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.857209  6486 consensus_queue.cc:260] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.857411  6484 consensus_queue.cc:260] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.859582  6486 ts_tablet_manager.cc:1434] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:15:01.860090  6484 ts_tablet_manager.cc:1434] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:15:01.861008  6486 tablet_bootstrap.cc:492] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:01.861109  6484 tablet_bootstrap.cc:492] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:01.862010  6426 raft_consensus.cc:493] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.871815  6426 raft_consensus.cc:515] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.873984  6484 tablet_bootstrap.cc:654] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.874478  6426 leader_election.cc:290] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:01.877490  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a10248efa3245f3afbde0526ce1d96d" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
W20260709 13:15:01.879936  6316 leader_election.cc:343] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:01.887051  6487 tablet_bootstrap.cc:654] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.887990  6484 tablet_bootstrap.cc:492] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:01.888646  6484 ts_tablet_manager.cc:1403] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 13:15:01.891709  6484 raft_consensus.cc:359] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.892455  6484 raft_consensus.cc:385] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.892633  6486 tablet_bootstrap.cc:654] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.892807  6484 raft_consensus.cc:740] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.893810  6484 consensus_queue.cc:260] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.896588  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a10248efa3245f3afbde0526ce1d96d" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:01.897483  6201 raft_consensus.cc:2468] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:15:01.898857  6314 leader_election.cc:304] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:01.899804  6495 raft_consensus.cc:2804] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:01.900235  6495 raft_consensus.cc:493] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.900241  6484 ts_tablet_manager.cc:1434] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:15:01.900595  6495 raft_consensus.cc:3060] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:01.901809  6487 tablet_bootstrap.cc:492] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:01.901952  6484 tablet_bootstrap.cc:492] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:01.902451  6487 ts_tablet_manager.cc:1403] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.006s
I20260709 13:15:01.905504  6487 raft_consensus.cc:359] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.906229  6487 raft_consensus.cc:385] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.906553  6487 raft_consensus.cc:740] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.907053  6486 tablet_bootstrap.cc:492] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:15:01.907652  6486 ts_tablet_manager.cc:1403] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.002s
I20260709 13:15:01.907480  6487 consensus_queue.cc:260] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.909161  6495 raft_consensus.cc:515] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.911870  6486 raft_consensus.cc:359] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.912246  6487 ts_tablet_manager.cc:1434] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:15:01.912227  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a10248efa3245f3afbde0526ce1d96d" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:01.912777  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a10248efa3245f3afbde0526ce1d96d" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:01.913053  6201 raft_consensus.cc:3060] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:01.913789  6487 tablet_bootstrap.cc:492] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
W20260709 13:15:01.914379  6316 leader_election.cc:343] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:01.915205  6495 leader_election.cc:290] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:01.912686  6486 raft_consensus.cc:385] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.916436  6486 raft_consensus.cc:740] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.917088  6484 tablet_bootstrap.cc:654] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.917335  6486 consensus_queue.cc:260] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.919764  6486 ts_tablet_manager.cc:1434] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:15:01.920919  6486 tablet_bootstrap.cc:492] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:01.921221  6487 tablet_bootstrap.cc:654] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.942663  6486 tablet_bootstrap.cc:654] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.951444  6201 raft_consensus.cc:2468] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:15:01.961663  6314 leader_election.cc:304] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:01.962859  6495 raft_consensus.cc:2804] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:01.963567  6495 raft_consensus.cc:697] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:01.964464  6495 consensus_queue.cc:237] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.966549  6484 tablet_bootstrap.cc:492] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:01.967141  6484 ts_tablet_manager.cc:1403] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.000s
I20260709 13:15:01.967342  6487 tablet_bootstrap.cc:492] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:01.967882  6487 ts_tablet_manager.cc:1403] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.054s	user 0.038s	sys 0.013s
I20260709 13:15:01.968546  6430 raft_consensus.cc:493] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.969092  6430 raft_consensus.cc:515] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.971249  6430 leader_election.cc:290] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:01.970978  6487 raft_consensus.cc:359] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.972082  6487 raft_consensus.cc:385] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.972397  6487 raft_consensus.cc:740] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.973075  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4299aff2799e405d8a924eb8227e6fdb" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:01.973202  6487 consensus_queue.cc:260] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
W20260709 13:15:01.975466  6242 leader_election.cc:343] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:01.975730  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4299aff2799e405d8a924eb8227e6fdb" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:01.976377  6487 ts_tablet_manager.cc:1434] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:01.977407  6487 tablet_bootstrap.cc:492] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:15:01.977514  6486 tablet_bootstrap.cc:492] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
W20260709 13:15:01.979619  6240 leader_election.cc:343] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:01.979091  6097 catalog_manager.cc:5697] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:01.980146  6240 leader_election.cc:304] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703
I20260709 13:15:01.980449  6486 ts_tablet_manager.cc:1403] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.060s	user 0.018s	sys 0.005s
I20260709 13:15:01.982293  6485 raft_consensus.cc:2749] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:01.970954  6484 raft_consensus.cc:359] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.983225  6484 raft_consensus.cc:385] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.983644  6484 raft_consensus.cc:740] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.984341  6487 tablet_bootstrap.cc:654] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:01.985894  6484 consensus_queue.cc:260] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.989005  6486 raft_consensus.cc:359] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.989933  6486 raft_consensus.cc:385] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:01.990226  6486 raft_consensus.cc:740] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:01.990952  6486 consensus_queue.cc:260] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.992177  6426 raft_consensus.cc:493] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.992789  6484 ts_tablet_manager.cc:1434] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.000s
I20260709 13:15:01.992630  6426 raft_consensus.cc:515] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:01.993471  6486 ts_tablet_manager.cc:1434] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:15:01.993902  6484 tablet_bootstrap.cc:492] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:01.994450  6486 tablet_bootstrap.cc:492] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:01.994736  6426 leader_election.cc:290] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:01.996932  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:15:01.997677  6277 raft_consensus.cc:2468] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:15:02.000469  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:02.001402  6316 leader_election.cc:304] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.002853  6495 raft_consensus.cc:2804] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:02.003357  6495 raft_consensus.cc:493] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.003785  6495 raft_consensus.cc:3060] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:02.004340  6314 leader_election.cc:343] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:02.011894  6486 tablet_bootstrap.cc:654] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.014201  6484 tablet_bootstrap.cc:654] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.015461  6487 tablet_bootstrap.cc:492] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:02.016036  6487 ts_tablet_manager.cc:1403] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.004s
I20260709 13:15:02.015854  6495 raft_consensus.cc:515] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.020296  6495 leader_election.cc:290] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:02.020577  6487 raft_consensus.cc:359] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.056600  6487 raft_consensus.cc:385] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.057049  6487 raft_consensus.cc:740] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.058545  6487 consensus_queue.cc:260] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.063361  6487 ts_tablet_manager.cc:1434] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.047s	user 0.003s	sys 0.007s
I20260709 13:15:02.070796  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:02.072086  6277 raft_consensus.cc:3060] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.079559  6484 tablet_bootstrap.cc:492] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:02.082840  6484 ts_tablet_manager.cc:1403] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.089s	user 0.017s	sys 0.003s
I20260709 13:15:02.081686  6487 tablet_bootstrap.cc:492] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:15:02.091529  6484 raft_consensus.cc:359] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.092595  6484 raft_consensus.cc:385] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.092955  6484 raft_consensus.cc:740] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.094165  6486 tablet_bootstrap.cc:492] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:15:02.093916  6484 consensus_queue.cc:260] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.095769  6486 ts_tablet_manager.cc:1403] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.102s	user 0.015s	sys 0.006s
I20260709 13:15:02.101904  6486 raft_consensus.cc:359] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.102802  6486 raft_consensus.cc:385] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.103147  6486 raft_consensus.cc:740] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.104341  6487 tablet_bootstrap.cc:654] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.104027  6486 consensus_queue.cc:260] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.109191  6484 ts_tablet_manager.cc:1434] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.010s
I20260709 13:15:02.105762  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:02.114810  6484 tablet_bootstrap.cc:492] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:02.111016  6277 raft_consensus.cc:2468] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:15:02.116369  6486 ts_tablet_manager.cc:1434] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.003s
W20260709 13:15:02.116811  6314 leader_election.cc:343] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:02.120605  6316 leader_election.cc:304] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:15:02.123827  6495 raft_consensus.cc:2804] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:02.124549  6495 raft_consensus.cc:697] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:02.126106  6495 consensus_queue.cc:237] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.128525  6486 tablet_bootstrap.cc:492] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:02.133630  6430 raft_consensus.cc:493] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.134613  6430 raft_consensus.cc:515] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.137991  6484 tablet_bootstrap.cc:654] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.138706  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113550147ed34c9ea2d337c2aca91f4e" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:02.149268  6201 raft_consensus.cc:2468] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:15:02.141974  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113550147ed34c9ea2d337c2aca91f4e" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:02.146345  6426 raft_consensus.cc:493] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.151398  6426 raft_consensus.cc:515] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.154502  6426 leader_election.cc:290] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:02.151618  6240 leader_election.cc:304] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.150666  6351 raft_consensus.cc:2468] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:15:02.138944  6430 leader_election.cc:290] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.156531  6430 raft_consensus.cc:2804] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:02.156898  6430 raft_consensus.cc:493] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.157236  6430 raft_consensus.cc:3060] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.158689  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99808726cdea4e4897076087e79c92f4" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:02.159789  6201 raft_consensus.cc:2468] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:15:02.160230  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99808726cdea4e4897076087e79c92f4" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:15:02.161952  6314 leader_election.cc:304] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: 
W20260709 13:15:02.163944  6316 leader_election.cc:343] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:02.167387  6426 raft_consensus.cc:2804] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:02.169055  6511 raft_consensus.cc:493] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.169937  6511 raft_consensus.cc:515] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.171173  6487 tablet_bootstrap.cc:492] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:02.171833  6487 ts_tablet_manager.cc:1403] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.009s
I20260709 13:15:02.173934  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f8b5015e29d44d4a5cbe2d00f415a18" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:15:02.174640  6275 raft_consensus.cc:2468] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 0.
I20260709 13:15:02.174947  6511 leader_election.cc:290] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.175158  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f8b5015e29d44d4a5cbe2d00f415a18" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:02.175817  6351 raft_consensus.cc:2468] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 0.
I20260709 13:15:02.175957  6167 leader_election.cc:304] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.168387  6426 raft_consensus.cc:493] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.181921  6426 raft_consensus.cc:3060] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.186105  6486 tablet_bootstrap.cc:654] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.188199  6511 raft_consensus.cc:2804] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:02.188664  6511 raft_consensus.cc:493] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.189031  6511 raft_consensus.cc:3060] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.192479  6487 raft_consensus.cc:359] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.193679  6487 raft_consensus.cc:385] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.194034  6487 raft_consensus.cc:740] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.195032  6487 consensus_queue.cc:260] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.200713  6486 tablet_bootstrap.cc:492] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:15:02.201325  6486 ts_tablet_manager.cc:1403] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.007s
I20260709 13:15:02.204097  6426 raft_consensus.cc:515] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.206773  6426 leader_election.cc:290] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:02.207585  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99808726cdea4e4897076087e79c92f4" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:02.208320  6201 raft_consensus.cc:3060] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.208185  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99808726cdea4e4897076087e79c92f4" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
W20260709 13:15:02.209678  6316 leader_election.cc:343] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:02.206789  6430 raft_consensus.cc:515] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.214670  6430 leader_election.cc:290] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.215553  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113550147ed34c9ea2d337c2aca91f4e" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:02.215821  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113550147ed34c9ea2d337c2aca91f4e" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.216295  6202 raft_consensus.cc:3060] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.216452  6351 raft_consensus.cc:3060] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.217896  6487 ts_tablet_manager.cc:1434] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 13:15:02.219408  6487 tablet_bootstrap.cc:492] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:15:02.225884  6484 tablet_bootstrap.cc:492] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:02.226522  6484 ts_tablet_manager.cc:1403] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.112s	user 0.015s	sys 0.004s
I20260709 13:15:02.226748  6487 tablet_bootstrap.cc:654] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.228809  6511 raft_consensus.cc:515] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.229982  6484 raft_consensus.cc:359] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.230752  6484 raft_consensus.cc:385] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.231022  6484 raft_consensus.cc:740] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.231882  6511 leader_election.cc:290] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.231788  6484 consensus_queue.cc:260] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.231797  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f8b5015e29d44d4a5cbe2d00f415a18" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:02.232856  6275 raft_consensus.cc:3060] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.234166  6484 ts_tablet_manager.cc:1434] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:02.235394  6484 tablet_bootstrap.cc:492] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:02.240031  6511 raft_consensus.cc:493] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.240209  6202 raft_consensus.cc:2468] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:15:02.240841  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f8b5015e29d44d4a5cbe2d00f415a18" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.240684  6511 raft_consensus.cc:515] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.241488  6349 raft_consensus.cc:3060] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.241978  6240 leader_election.cc:304] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.243046  6430 raft_consensus.cc:2804] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:02.243518  6430 raft_consensus.cc:697] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:02.243835  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:15:02.239687  6486 raft_consensus.cc:359] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.244815  6486 raft_consensus.cc:385] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.244621  6430 consensus_queue.cc:237] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
W20260709 13:15:02.245575  6167 leader_election.cc:343] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:02.246698  6511 leader_election.cc:290] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.245178  6486 raft_consensus.cc:740] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.248551  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:02.249300  6351 raft_consensus.cc:2468] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:15:02.249709  6486 consensus_queue.cc:260] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.250582  6484 tablet_bootstrap.cc:654] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.252215  6350 raft_consensus.cc:2468] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 0.
I20260709 13:15:02.254704  6167 leader_election.cc:304] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.255501  6275 raft_consensus.cc:2468] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 1.
I20260709 13:15:02.256356  6511 raft_consensus.cc:2804] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:02.256783  6167 leader_election.cc:304] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.256916  6511 raft_consensus.cc:493] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.257552  6511 raft_consensus.cc:3060] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.258080  6508 raft_consensus.cc:2804] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:02.258916  6508 raft_consensus.cc:697] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:02.260257  6486 ts_tablet_manager.cc:1434] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.004s
I20260709 13:15:02.259994  6508 consensus_queue.cc:237] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.265931  6201 raft_consensus.cc:2468] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:15:02.267601  6486 tablet_bootstrap.cc:492] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:02.275172  6314 leader_election.cc:304] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.276248  6484 tablet_bootstrap.cc:492] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:02.277307  6426 raft_consensus.cc:2804] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:02.277321  6484 ts_tablet_manager.cc:1403] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.015s
I20260709 13:15:02.277877  6426 raft_consensus.cc:697] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:02.278864  6426 consensus_queue.cc:237] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.275156  6511 raft_consensus.cc:515] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.276257  6487 tablet_bootstrap.cc:492] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:02.281351  6487 ts_tablet_manager.cc:1403] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.008s
I20260709 13:15:02.283223  6484 raft_consensus.cc:359] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.284097  6484 raft_consensus.cc:385] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.284368  6484 raft_consensus.cc:740] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.285132  6484 consensus_queue.cc:260] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.285288  6099 catalog_manager.cc:5697] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 0 to 1, leader changed from <none> to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 1 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.285738  6487 raft_consensus.cc:359] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.286775  6484 ts_tablet_manager.cc:1434] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:15:02.287089  6487 raft_consensus.cc:385] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.287523  6487 raft_consensus.cc:740] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.288085  6484 tablet_bootstrap.cc:492] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0: Bootstrap starting.
I20260709 13:15:02.288656  6487 consensus_queue.cc:260] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.298874  6486 tablet_bootstrap.cc:654] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.301507  6487 ts_tablet_manager.cc:1434] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:15:02.302970  6487 tablet_bootstrap.cc:492] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:15:02.308337  6484 tablet_bootstrap.cc:654] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.308483  6511 leader_election.cc:290] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.304206  6349 raft_consensus.cc:2468] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 1.
I20260709 13:15:02.312443  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
W20260709 13:15:02.314462  6167 leader_election.cc:343] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:02.316530  6487 tablet_bootstrap.cc:654] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.324302  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.325186  6349 raft_consensus.cc:3060] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.326587  6099 catalog_manager.cc:5697] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.331523  6484 tablet_bootstrap.cc:492] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0: No bootstrap required, opened a new log
I20260709 13:15:02.332152  6484 ts_tablet_manager.cc:1403] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.014s
I20260709 13:15:02.332490  6486 tablet_bootstrap.cc:492] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:15:02.333071  6486 ts_tablet_manager.cc:1403] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.001s
I20260709 13:15:02.333523  6349 raft_consensus.cc:2468] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 1.
I20260709 13:15:02.335800  6487 tablet_bootstrap.cc:492] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:02.335961  6167 leader_election.cc:304] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.336383  6487 ts_tablet_manager.cc:1403] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.006s
I20260709 13:15:02.336483  6511 raft_consensus.cc:493] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.336831  6508 raft_consensus.cc:2804] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:02.337041  6511 raft_consensus.cc:515] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.336158  6486 raft_consensus.cc:359] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.338194  6486 raft_consensus.cc:385] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.338491  6486 raft_consensus.cc:740] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.337397  6508 raft_consensus.cc:697] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:02.339246  6511 leader_election.cc:290] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.339221  6487 raft_consensus.cc:359] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.339963  6487 raft_consensus.cc:385] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.340384  6487 raft_consensus.cc:740] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.339231  6486 consensus_queue.cc:260] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.341306  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:02.341866  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:15:02.342733  6275 raft_consensus.cc:2393] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16719ad7c0bd4042acd734beca09a9e0 in current term 1: Already voted for candidate b3ae2edef6a0467eaf647236a2c51703 in this term.
I20260709 13:15:02.342422  6508 consensus_queue.cc:237] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.343676  6486 ts_tablet_manager.cc:1434] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:15:02.343976  6167 leader_election.cc:304] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0; no voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.344928  6511 raft_consensus.cc:3060] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.345137  6486 tablet_bootstrap.cc:492] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:02.345209  6097 catalog_manager.cc:5697] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.335435  6484 raft_consensus.cc:359] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.347319  6484 raft_consensus.cc:385] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.348109  6484 raft_consensus.cc:740] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.349259  6484 consensus_queue.cc:260] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.352859  6484 ts_tablet_manager.cc:1434] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:15:02.353554  6511 raft_consensus.cc:2749] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:02.343333  6487 consensus_queue.cc:260] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.357533  6487 ts_tablet_manager.cc:1434] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.002s
I20260709 13:15:02.358168  6486 tablet_bootstrap.cc:654] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.358641  6487 tablet_bootstrap.cc:492] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba: Bootstrap starting.
I20260709 13:15:02.369743  6099 catalog_manager.cc:5697] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.371819  6487 tablet_bootstrap.cc:654] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.380681  6487 tablet_bootstrap.cc:492] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba: No bootstrap required, opened a new log
I20260709 13:15:02.381136  6487 ts_tablet_manager.cc:1403] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.004s
I20260709 13:15:02.382920  6275 raft_consensus.cc:3060] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.384277  6487 raft_consensus.cc:359] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.385164  6487 raft_consensus.cc:385] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.385437  6487 raft_consensus.cc:740] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.386340  6487 consensus_queue.cc:260] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.388813  6487 ts_tablet_manager.cc:1434] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:15:02.396765  6486 tablet_bootstrap.cc:492] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:15:02.397522  6486 ts_tablet_manager.cc:1403] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.053s	user 0.027s	sys 0.016s
I20260709 13:15:02.399314  6426 consensus_queue.cc:1048] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:02.402133  6486 raft_consensus.cc:359] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.403234  6486 raft_consensus.cc:385] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.402643  6098 catalog_manager.cc:5697] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.403605  6486 raft_consensus.cc:740] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.404647  6486 consensus_queue.cc:260] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.407608  6486 ts_tablet_manager.cc:1434] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:15:02.409148  6486 tablet_bootstrap.cc:492] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703: Bootstrap starting.
I20260709 13:15:02.416208  6486 tablet_bootstrap.cc:654] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:02.417384  6426 consensus_queue.cc:1048] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.442865  6430 raft_consensus.cc:493] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.443394  6430 raft_consensus.cc:515] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.446496  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99808726cdea4e4897076087e79c92f4" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:02.447496  6202 raft_consensus.cc:2393] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd18efee76df4c2699bff236b9d056ba in current term 1: Already voted for candidate b3ae2edef6a0467eaf647236a2c51703 in this term.
I20260709 13:15:02.449887  6430 leader_election.cc:290] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.451228  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99808726cdea4e4897076087e79c92f4" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:02.453871  6242 leader_election.cc:304] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703
I20260709 13:15:02.455605  6430 raft_consensus.cc:3060] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.458273  6486 tablet_bootstrap.cc:492] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703: No bootstrap required, opened a new log
I20260709 13:15:02.458714  6486 ts_tablet_manager.cc:1403] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703: Time spent bootstrapping tablet: real 0.050s	user 0.032s	sys 0.012s
I20260709 13:15:02.460928  6486 raft_consensus.cc:359] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.461449  6486 raft_consensus.cc:385] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:02.461683  6486 raft_consensus.cc:740] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Initialized, Role: FOLLOWER
I20260709 13:15:02.462303  6486 consensus_queue.cc:260] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.463920  6486 ts_tablet_manager.cc:1434] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.002s
I20260709 13:15:02.467087  6430 raft_consensus.cc:2749] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:02.484468  6430 raft_consensus.cc:493] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.485080  6430 raft_consensus.cc:515] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.486850  6430 leader_election.cc:290] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.487454  6514 raft_consensus.cc:493] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.487505  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:02.487843  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:02.488776  6349 raft_consensus.cc:2393] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd18efee76df4c2699bff236b9d056ba in current term 1: Already voted for candidate 16719ad7c0bd4042acd734beca09a9e0 in this term.
I20260709 13:15:02.488451  6514 raft_consensus.cc:515] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.489771  6242 leader_election.cc:304] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd18efee76df4c2699bff236b9d056ba; no voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703
I20260709 13:15:02.490463  6514 leader_election.cc:290] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:02.490890  6430 raft_consensus.cc:3060] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.491146  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "875130b8f7db4b0faec1c151bc3cf694" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:02.491484  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "875130b8f7db4b0faec1c151bc3cf694" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:15:02.491886  6201 raft_consensus.cc:2468] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:15:02.492123  6277 raft_consensus.cc:2468] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 0.
I20260709 13:15:02.492978  6314 leader_election.cc:304] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: 
I20260709 13:15:02.493706  6514 raft_consensus.cc:2804] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:02.494076  6514 raft_consensus.cc:493] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.494345  6514 raft_consensus.cc:3060] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.497195  6430 raft_consensus.cc:2749] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:02.499969  6514 raft_consensus.cc:515] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.501861  6514 leader_election.cc:290] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:02.502552  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "875130b8f7db4b0faec1c151bc3cf694" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:02.502821  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "875130b8f7db4b0faec1c151bc3cf694" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:02.503070  6201 raft_consensus.cc:3060] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.503376  6277 raft_consensus.cc:3060] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.509368  6277 raft_consensus.cc:2468] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:15:02.509711  6201 raft_consensus.cc:2468] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 1.
I20260709 13:15:02.510347  6316 leader_election.cc:304] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.511114  6514 raft_consensus.cc:2804] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:02.511516  6514 raft_consensus.cc:697] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:02.512269  6514 consensus_queue.cc:237] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.519407  6100 catalog_manager.cc:5697] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 1 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.530092  6485 raft_consensus.cc:493] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.530692  6485 raft_consensus.cc:515] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.532691  6485 leader_election.cc:290] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.533212  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824e6d3a7170481b9c540a57c7768843" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0" is_pre_election: true
I20260709 13:15:02.533749  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824e6d3a7170481b9c540a57c7768843" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:02.533862  6201 raft_consensus.cc:2468] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:15:02.534529  6349 raft_consensus.cc:2468] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 0.
I20260709 13:15:02.535040  6240 leader_election.cc:304] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.535856  6430 raft_consensus.cc:2804] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:02.536188  6430 raft_consensus.cc:493] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.536438  6430 raft_consensus.cc:3060] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.541496  6430 raft_consensus.cc:515] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.543009  6430 leader_election.cc:290] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.543823  6201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824e6d3a7170481b9c540a57c7768843" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:02.544102  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824e6d3a7170481b9c540a57c7768843" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.544437  6201 raft_consensus.cc:3060] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.544750  6349 raft_consensus.cc:3060] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.547314  6514 consensus_queue.cc:1048] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.551123  6201 raft_consensus.cc:2468] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:15:02.551555  6349 raft_consensus.cc:2468] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 1.
I20260709 13:15:02.552471  6240 leader_election.cc:304] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.553978  6430 raft_consensus.cc:2804] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:02.554499  6430 raft_consensus.cc:697] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:02.555326  6430 consensus_queue.cc:237] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.562563  6495 consensus_queue.cc:1048] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.568679  6100 catalog_manager.cc:5697] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 0 to 1, leader changed from <none> to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 1 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.615796  6511 raft_consensus.cc:493] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.616412  6511 raft_consensus.cc:515] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.618312  6511 leader_election.cc:290] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.619043  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4299aff2799e405d8a924eb8227e6fdb" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba" is_pre_election: true
I20260709 13:15:02.619310  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4299aff2799e405d8a924eb8227e6fdb" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703" is_pre_election: true
I20260709 13:15:02.619668  6277 raft_consensus.cc:2468] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 0.
I20260709 13:15:02.620177  6349 raft_consensus.cc:2468] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 0.
I20260709 13:15:02.620821  6167 leader_election.cc:304] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.621701  6511 raft_consensus.cc:2804] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:02.622058  6511 raft_consensus.cc:493] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.622347  6511 raft_consensus.cc:3060] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.628337  6511 raft_consensus.cc:515] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.629876  6511 leader_election.cc:290] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.630592  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4299aff2799e405d8a924eb8227e6fdb" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:02.630754  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4299aff2799e405d8a924eb8227e6fdb" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.631111  6277 raft_consensus.cc:3060] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.631319  6349 raft_consensus.cc:3060] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:02.636448  6277 raft_consensus.cc:2468] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 1.
I20260709 13:15:02.637485  6167 leader_election.cc:304] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.637723  6349 raft_consensus.cc:2468] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 1.
I20260709 13:15:02.638142  6511 raft_consensus.cc:2804] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:02.638721  6511 raft_consensus.cc:697] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:02.639487  6511 consensus_queue.cc:237] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.646968  6098 catalog_manager.cc:5697] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 1 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.684912  6511 consensus_queue.cc:1048] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.698659  6508 consensus_queue.cc:1048] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.698755  6430 consensus_queue.cc:1048] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:02.713081  6430 consensus_queue.cc:1048] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.727928  6277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "113550147ed34c9ea2d337c2aca91f4e"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
mode: GRACEFUL
new_leader_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:42888
I20260709 13:15:02.728683  6277 raft_consensus.cc:606] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Received request to transfer leadership to TS 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:15:02.733948  6350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99808726cdea4e4897076087e79c92f4"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
mode: GRACEFUL
new_leader_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:45564
I20260709 13:15:02.734440  6350 raft_consensus.cc:606] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Received request to transfer leadership to TS 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:15:02.739339  6350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a10248efa3245f3afbde0526ce1d96d"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
mode: GRACEFUL
new_leader_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:45564
I20260709 13:15:02.739823  6350 raft_consensus.cc:606] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Received request to transfer leadership to TS 16719ad7c0bd4042acd734beca09a9e0
I20260709 13:15:02.746366  6350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
mode: GRACEFUL
new_leader_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:45564
I20260709 13:15:02.746865  6350 raft_consensus.cc:606] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Received request to transfer leadership to TS dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.761595  6201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f8b5015e29d44d4a5cbe2d00f415a18"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
mode: GRACEFUL
new_leader_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:54912
I20260709 13:15:02.762125  6201 raft_consensus.cc:606] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Received request to transfer leadership to TS dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.767467  6350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "875130b8f7db4b0faec1c151bc3cf694"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
mode: GRACEFUL
new_leader_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:45564
I20260709 13:15:02.768011  6350 raft_consensus.cc:606] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Received request to transfer leadership to TS dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.771039  6508 consensus_queue.cc:1048] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.773214  6201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
mode: GRACEFUL
new_leader_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:54912
I20260709 13:15:02.773706  6201 raft_consensus.cc:606] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Received request to transfer leadership to TS dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.779667  6277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "824e6d3a7170481b9c540a57c7768843"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
mode: GRACEFUL
new_leader_uuid: "b3ae2edef6a0467eaf647236a2c51703"
 from {username='slave'} at 127.0.0.1:42888
I20260709 13:15:02.780249  6277 raft_consensus.cc:606] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Received request to transfer leadership to TS b3ae2edef6a0467eaf647236a2c51703
I20260709 13:15:02.782011  6452 consensus_queue.cc:1048] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.792518  6452 raft_consensus.cc:993] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0: : Instructing follower dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:15:02.792991  6511 raft_consensus.cc:1081] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Signalling peer dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:15:02.794524  6275 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:42866
I20260709 13:15:02.795153  6275 raft_consensus.cc:493] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:02.795516  6275 raft_consensus.cc:3060] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.804208  6495 consensus_queue.cc:1048] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:02.804476  6275 raft_consensus.cc:515] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.807328  6275 leader_election.cc:290] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.807534  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.807863  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36330902a8014b269c0ed6ffaf54c238" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:02.808833  6202 raft_consensus.cc:3055] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:02.809290  6202 raft_consensus.cc:740] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:02.810356  6202 consensus_queue.cc:260] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.808252  6350 raft_consensus.cc:3060] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.812145  6202 raft_consensus.cc:3060] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.821838  6350 raft_consensus.cc:2468] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:15:02.822728  6495 raft_consensus.cc:993] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703: : Instructing follower 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:15:02.823052  6242 leader_election.cc:304] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.823246  6495 raft_consensus.cc:1081] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Signalling peer 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:15:02.823993  6542 raft_consensus.cc:2804] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:02.824496  6542 raft_consensus.cc:697] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:02.825317  6542 consensus_queue.cc:237] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.830473  6200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99808726cdea4e4897076087e79c92f4"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:53102
I20260709 13:15:02.831593  6200 raft_consensus.cc:493] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:02.831970  6200 raft_consensus.cc:3060] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.835000  6100 catalog_manager.cc:5697] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 1 to 2, leader changed from 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65) to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 2 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.837145  6202 raft_consensus.cc:2468] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:15:02.840222  6514 consensus_queue.cc:1048] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.841914  6200 raft_consensus.cc:515] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.846443  6200 leader_election.cc:290] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.846982  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99808726cdea4e4897076087e79c92f4" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:02.849231  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99808726cdea4e4897076087e79c92f4" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.849989  6350 raft_consensus.cc:3055] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:02.850378  6350 raft_consensus.cc:740] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:02.851339  6350 consensus_queue.cc:260] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.853034  6350 raft_consensus.cc:3060] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.861897  6350 raft_consensus.cc:2468] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:15:02.862953  6167 leader_election.cc:304] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: dd18efee76df4c2699bff236b9d056ba
I20260709 13:15:02.863626  6511 raft_consensus.cc:2804] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:02.864176  6511 raft_consensus.cc:697] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:02.865051  6511 consensus_queue.cc:237] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.872649  6099 catalog_manager.cc:5697] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 1 to 2, leader changed from b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67) to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.934405  6514 consensus_queue.cc:1048] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.944032  6495 consensus_queue.cc:1048] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:02.953322  6495 raft_consensus.cc:993] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703: : Instructing follower dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:15:02.953815  6426 raft_consensus.cc:1081] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Signalling peer dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:15:02.955363  6277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "875130b8f7db4b0faec1c151bc3cf694"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:42874
I20260709 13:15:02.955856  6277 raft_consensus.cc:493] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:02.956197  6277 raft_consensus.cc:3060] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.957804  6426 raft_consensus.cc:993] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: : Instructing follower dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:15:02.958349  6426 raft_consensus.cc:1081] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Signalling peer dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:15:02.960028  6275 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:42874
I20260709 13:15:02.960606  6275 raft_consensus.cc:493] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:02.960947  6275 raft_consensus.cc:3060] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.963145  6514 raft_consensus.cc:993] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: : Instructing follower 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:15:02.963694  6426 raft_consensus.cc:1081] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Signalling peer 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:15:02.963939  6277 raft_consensus.cc:515] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.966146  6277 leader_election.cc:290] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.967235  6350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "875130b8f7db4b0faec1c151bc3cf694" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.967296  6200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "875130b8f7db4b0faec1c151bc3cf694" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:02.967953  6350 raft_consensus.cc:3055] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:02.968369  6350 raft_consensus.cc:740] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:02.968308  6200 raft_consensus.cc:3060] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.968371  6202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a10248efa3245f3afbde0526ce1d96d"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:53102
I20260709 13:15:02.969384  6202 raft_consensus.cc:493] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:02.969527  6350 consensus_queue.cc:260] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.969842  6202 raft_consensus.cc:3060] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.970708  6350 raft_consensus.cc:3060] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.976421  6202 raft_consensus.cc:515] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.977468  6350 raft_consensus.cc:2468] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:15:02.978581  6202 leader_election.cc:290] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.978711  6242 leader_election.cc:304] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.979573  6542 raft_consensus.cc:2804] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:02.980036  6542 raft_consensus.cc:697] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:02.980595  6275 raft_consensus.cc:515] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.980903  6542 consensus_queue.cc:237] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.981455  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a10248efa3245f3afbde0526ce1d96d" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.982245  6349 raft_consensus.cc:3055] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:02.982600  6349 raft_consensus.cc:740] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:02.983140  6275 leader_election.cc:290] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:02.983634  6349 consensus_queue.cc:260] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.984241  6202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:02.984872  6202 raft_consensus.cc:3060] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.984925  6349 raft_consensus.cc:3060] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.991178  6349 raft_consensus.cc:2468] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:15:02.992062  6202 raft_consensus.cc:2468] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:15:02.993257  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a10248efa3245f3afbde0526ce1d96d" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:02.993427  6240 leader_election.cc:304] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:02.993916  6275 raft_consensus.cc:3060] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.994081  6351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6a8dd7983645259cbb2cc1f09c6e7c" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:02.994365  6542 raft_consensus.cc:2804] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:02.994688  6351 raft_consensus.cc:3055] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:02.995070  6542 raft_consensus.cc:697] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:02.995131  6351 raft_consensus.cc:740] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:02.996121  6351 consensus_queue.cc:260] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:02.997315  6351 raft_consensus.cc:3060] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.996807  6100 catalog_manager.cc:5697] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 1 to 2, leader changed from b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67) to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 2 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.994556  6167 leader_election.cc:304] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: 
I20260709 13:15:03.000387  6511 raft_consensus.cc:2804] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:03.000885  6511 raft_consensus.cc:697] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:03.002056  6511 consensus_queue.cc:237] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.000850  6542 consensus_queue.cc:237] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.009049  6351 raft_consensus.cc:2468] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:15:03.005882  6200 raft_consensus.cc:2468] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:15:03.007634  6275 raft_consensus.cc:2468] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:15:03.012869  6099 catalog_manager.cc:5697] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 1 to 2, leader changed from b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67) to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:03.016709  6100 catalog_manager.cc:5697] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 1 to 2, leader changed from b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67) to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 2 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:03.036556  6542 consensus_queue.cc:1048] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:03.043040  6485 raft_consensus.cc:993] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba: : Instructing follower b3ae2edef6a0467eaf647236a2c51703 to start an election
I20260709 13:15:03.043594  6485 raft_consensus.cc:1081] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Signalling peer b3ae2edef6a0467eaf647236a2c51703 to start an election
I20260709 13:15:03.045380  6350 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "824e6d3a7170481b9c540a57c7768843"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
 from {username='slave'} at 127.0.0.1:45550
I20260709 13:15:03.045950  6350 raft_consensus.cc:493] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:03.046339  6350 raft_consensus.cc:3060] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.048930  6542 consensus_queue.cc:1048] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:03.053475  6350 raft_consensus.cc:515] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.054706  6349 raft_consensus.cc:1240] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Rejecting Update request from peer dd18efee76df4c2699bff236b9d056ba for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:03.058594  6542 consensus_queue.cc:1059] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:03.059223  6485 raft_consensus.cc:3055] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:03.059577  6485 raft_consensus.cc:740] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:03.060379  6485 consensus_queue.cc:260] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.060909  6350 leader_election.cc:290] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 2 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:03.061508  6485 raft_consensus.cc:3060] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.061767  6200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824e6d3a7170481b9c540a57c7768843" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:03.062393  6200 raft_consensus.cc:3060] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.063896  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "824e6d3a7170481b9c540a57c7768843" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:03.070188  6200 raft_consensus.cc:2468] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 2.
I20260709 13:15:03.071324  6314 leader_election.cc:304] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: 
I20260709 13:15:03.072081  6426 raft_consensus.cc:2804] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:03.072587  6426 raft_consensus.cc:697] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 2 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:03.073397  6426 consensus_queue.cc:237] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.076122  6275 raft_consensus.cc:2468] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 2.
I20260709 13:15:03.081485  6100 catalog_manager.cc:5697] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 1 to 2, leader changed from dd18efee76df4c2699bff236b9d056ba (127.27.63.66) to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 2 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:03.111506  6511 raft_consensus.cc:993] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0: : Instructing follower dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:15:03.112067  6508 raft_consensus.cc:1081] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Signalling peer dd18efee76df4c2699bff236b9d056ba to start an election
I20260709 13:15:03.113519  6275 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f8b5015e29d44d4a5cbe2d00f415a18"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
 from {username='slave'} at 127.0.0.1:42866
I20260709 13:15:03.114010  6275 raft_consensus.cc:493] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:03.114297  6275 raft_consensus.cc:3060] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.119453  6275 raft_consensus.cc:515] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.121119  6275 leader_election.cc:290] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:03.122087  6200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f8b5015e29d44d4a5cbe2d00f415a18" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:03.122114  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f8b5015e29d44d4a5cbe2d00f415a18" candidate_uuid: "dd18efee76df4c2699bff236b9d056ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:03.123235  6200 raft_consensus.cc:3055] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:03.123245  6349 raft_consensus.cc:3060] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.123708  6200 raft_consensus.cc:740] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:03.124565  6200 consensus_queue.cc:260] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.125844  6200 raft_consensus.cc:3060] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.130851  6349 raft_consensus.cc:2468] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:15:03.132225  6242 leader_election.cc:304] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:03.133018  6485 raft_consensus.cc:2804] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:03.133052  6200 raft_consensus.cc:2468] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd18efee76df4c2699bff236b9d056ba in term 2.
I20260709 13:15:03.133790  6485 raft_consensus.cc:697] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Becoming Leader. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:03.134609  6485 consensus_queue.cc:237] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.140805  6100 catalog_manager.cc:5697] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba reported cstate change: term changed from 1 to 2, leader changed from 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65) to dd18efee76df4c2699bff236b9d056ba (127.27.63.66). New cstate: current_term: 2 leader_uuid: "dd18efee76df4c2699bff236b9d056ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:03.154810  6508 consensus_queue.cc:1048] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:03.165391  6511 consensus_queue.cc:1048] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:03.266615  6542 raft_consensus.cc:993] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba: : Instructing follower 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:15:03.267104  6542 raft_consensus.cc:1081] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Signalling peer 16719ad7c0bd4042acd734beca09a9e0 to start an election
I20260709 13:15:03.268759  6200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "113550147ed34c9ea2d337c2aca91f4e"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
 from {username='slave'} at 127.0.0.1:53104
I20260709 13:15:03.269358  6200 raft_consensus.cc:493] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:03.269748  6200 raft_consensus.cc:3060] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.275225  6200 raft_consensus.cc:515] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.277206  6200 leader_election.cc:290] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Requested vote from peers dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991), b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67:43321)
I20260709 13:15:03.278023  6275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113550147ed34c9ea2d337c2aca91f4e" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:03.278291  6349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113550147ed34c9ea2d337c2aca91f4e" candidate_uuid: "16719ad7c0bd4042acd734beca09a9e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
I20260709 13:15:03.278817  6275 raft_consensus.cc:3055] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:03.278883  6349 raft_consensus.cc:3060] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.279178  6275 raft_consensus.cc:740] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:03.280140  6275 consensus_queue.cc:260] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.281205  6275 raft_consensus.cc:3060] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.284720  6349 raft_consensus.cc:2468] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:15:03.285790  6167 leader_election.cc:304] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: 
I20260709 13:15:03.286504  6511 raft_consensus.cc:2804] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:03.286892  6511 raft_consensus.cc:697] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Becoming Leader. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:03.287607  6275 raft_consensus.cc:2468] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16719ad7c0bd4042acd734beca09a9e0 in term 2.
I20260709 13:15:03.287715  6511 consensus_queue.cc:237] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:03.294992  6099 catalog_manager.cc:5697] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 reported cstate change: term changed from 1 to 2, leader changed from dd18efee76df4c2699bff236b9d056ba (127.27.63.66) to 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65). New cstate: current_term: 2 leader_uuid: "16719ad7c0bd4042acd734beca09a9e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:03.306242  6277 raft_consensus.cc:3060] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:03.311564  6277 raft_consensus.cc:1275] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.312840  6511 consensus_queue.cc:1048] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.320314  6349 raft_consensus.cc:1275] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.322438  6508 consensus_queue.cc:1048] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:03.354713  6349 raft_consensus.cc:1275] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.356312  6485 consensus_queue.cc:1048] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.365046  6200 raft_consensus.cc:1275] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.366899  6542 consensus_queue.cc:1048] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.392848  6349 raft_consensus.cc:1275] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.394093  6508 consensus_queue.cc:1048] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.401409  6277 raft_consensus.cc:1275] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.403753  6200 raft_consensus.cc:1275] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.404702  6452 consensus_queue.cc:1048] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.405253  6542 consensus_queue.cc:1048] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.415117  6349 raft_consensus.cc:1275] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.416601  6430 consensus_queue.cc:1048] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.459856  6349 raft_consensus.cc:1275] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.461292  6430 consensus_queue.cc:1048] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.472318  6200 raft_consensus.cc:1275] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.473685  6485 consensus_queue.cc:1048] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.556430  6349 raft_consensus.cc:1275] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.557803  6485 consensus_queue.cc:1048] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.562018  6202 raft_consensus.cc:1275] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Refusing update from remote peer b3ae2edef6a0467eaf647236a2c51703: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.563563  6495 consensus_queue.cc:1048] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.571201  6202 raft_consensus.cc:1275] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Refusing update from remote peer dd18efee76df4c2699bff236b9d056ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.573803  6430 consensus_queue.cc:1048] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.581542  6277 raft_consensus.cc:1275] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Refusing update from remote peer b3ae2edef6a0467eaf647236a2c51703: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.583309  6557 consensus_queue.cc:1048] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.826280  6350 raft_consensus.cc:1275] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.827669  6511 consensus_queue.cc:1048] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.838085  6277 raft_consensus.cc:1275] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Refusing update from remote peer 16719ad7c0bd4042acd734beca09a9e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.839408  6452 consensus_queue.cc:1048] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:05.784400  6098 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36346:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:15:05.806202  6098 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36346:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:15:05.837932 27901 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:05.975730  6596 log.cc:927] Time spent T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:06.089488  6613 tablet.cc:1724] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.165642  6621 tablet.cc:1724] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.166563  6628 tablet.cc:1724] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.185322  6619 tablet.cc:1724] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:15:06.181735  6607 log.cc:927] Time spent T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:15:06.204099  6618 tablet.cc:1724] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.247053  6633 tablet.cc:1724] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.251156  6625 tablet.cc:1724] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.263931  6630 tablet.cc:1724] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.279860  6621 tablet.cc:1724] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.292191  6277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2"
dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
mode: GRACEFUL
new_leader_uuid: "b3ae2edef6a0467eaf647236a2c51703"
 from {username='slave'} at 127.0.0.1:49886
I20260709 13:15:06.292867  6277 raft_consensus.cc:606] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Received request to transfer leadership to TS b3ae2edef6a0467eaf647236a2c51703
I20260709 13:15:06.304819  6637 tablet.cc:1724] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.283277  6613 tablet.cc:1724] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.305728  6634 tablet.cc:1724] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.310190  6618 tablet.cc:1724] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.305207  6619 tablet.cc:1724] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.333148  6630 tablet.cc:1724] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.344556  6634 tablet.cc:1724] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.303575  6635 tablet.cc:1724] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.349627  6628 tablet.cc:1724] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.340219  6632 tablet.cc:1724] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.372699  6630 tablet.cc:1724] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.375098  6621 tablet.cc:1724] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.384132  6618 tablet.cc:1724] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.384972  6634 tablet.cc:1724] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.398566  6632 tablet.cc:1724] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.410272  6198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051"
dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
mode: GRACEFUL
new_leader_uuid: "b3ae2edef6a0467eaf647236a2c51703"
 from {username='slave'} at 127.0.0.1:54924
I20260709 13:15:06.411011  6198 raft_consensus.cc:606] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Received request to transfer leadership to TS b3ae2edef6a0467eaf647236a2c51703
I20260709 13:15:06.412691 27901 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:06.413313 27901 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:06.414757  6625 tablet.cc:1724] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.403703  6633 tablet.cc:1724] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.421202  6635 tablet.cc:1724] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:15:06.459880  6098 catalog_manager.cc:6242] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2: soft_deleted_table [id=7188dca5516e4b128ce99a6a55cc60e9] alter complete (version 1)
I20260709 13:15:06.571049  6564 raft_consensus.cc:993] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0: : Instructing follower b3ae2edef6a0467eaf647236a2c51703 to start an election
I20260709 13:15:06.571695  6563 raft_consensus.cc:1081] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Signalling peer b3ae2edef6a0467eaf647236a2c51703 to start an election
I20260709 13:15:06.573436  6350 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
 from {username='slave'} at 127.0.0.1:45544
I20260709 13:15:06.573974  6350 raft_consensus.cc:493] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:06.574321  6350 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:06.581866  6350 raft_consensus.cc:515] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:06.583531  6350 leader_election.cc:290] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 3 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:06.584412  6198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:06.584725  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c41a951e84424e2dbc8f7994aeece051" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:06.585054  6198 raft_consensus.cc:3055] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:06.585364  6277 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:06.585399  6198 raft_consensus.cc:740] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 16719ad7c0bd4042acd734beca09a9e0, State: Running, Role: LEADER
I20260709 13:15:06.586318  6198 consensus_queue.cc:260] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:06.587505  6198 raft_consensus.cc:3060] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:06.591362  6277 raft_consensus.cc:2468] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 3.
I20260709 13:15:06.592548  6316 leader_election.cc:304] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ae2edef6a0467eaf647236a2c51703, dd18efee76df4c2699bff236b9d056ba; no voters: 
I20260709 13:15:06.593144  6198 raft_consensus.cc:2468] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 3.
I20260709 13:15:06.593436  6614 raft_consensus.cc:2804] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:06.594003  6614 raft_consensus.cc:697] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 3 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:06.594830  6614 consensus_queue.cc:237] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:06.603480  6100 catalog_manager.cc:5697] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 2 to 3, leader changed from 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65) to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:06.834286  6606 raft_consensus.cc:993] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba: : Instructing follower b3ae2edef6a0467eaf647236a2c51703 to start an election
I20260709 13:15:06.834808  6616 raft_consensus.cc:1081] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Signalling peer b3ae2edef6a0467eaf647236a2c51703 to start an election
I20260709 13:15:06.836365  6351 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2"
dest_uuid: "b3ae2edef6a0467eaf647236a2c51703"
 from {username='slave'} at 127.0.0.1:45550
I20260709 13:15:06.836824  6351 raft_consensus.cc:493] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:06.837076  6351 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:06.843307  6351 raft_consensus.cc:515] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:06.844725  6351 leader_election.cc:290] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 3 election: Requested vote from peers 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369), dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991)
I20260709 13:15:06.845660  6198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "16719ad7c0bd4042acd734beca09a9e0"
I20260709 13:15:06.845934  6277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba75b94277b4e21adc71d3343f6f0f2" candidate_uuid: "b3ae2edef6a0467eaf647236a2c51703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd18efee76df4c2699bff236b9d056ba"
I20260709 13:15:06.846329  6198 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:06.846546  6277 raft_consensus.cc:3055] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:06.846911  6277 raft_consensus.cc:740] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd18efee76df4c2699bff236b9d056ba, State: Running, Role: LEADER
I20260709 13:15:06.847569  6277 consensus_queue.cc:260] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:06.848498  6277 raft_consensus.cc:3060] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:06.852180  6198 raft_consensus.cc:2468] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 3.
I20260709 13:15:06.853188  6277 raft_consensus.cc:2468] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ae2edef6a0467eaf647236a2c51703 in term 3.
I20260709 13:15:06.853643  6314 leader_election.cc:304] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16719ad7c0bd4042acd734beca09a9e0, b3ae2edef6a0467eaf647236a2c51703; no voters: 
I20260709 13:15:06.854662  6614 raft_consensus.cc:2804] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:06.855104  6614 raft_consensus.cc:697] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 3 LEADER]: Becoming Leader. State: Replica: b3ae2edef6a0467eaf647236a2c51703, State: Running, Role: LEADER
I20260709 13:15:06.855867  6614 consensus_queue.cc:237] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } }
I20260709 13:15:06.864560  6100 catalog_manager.cc:5697] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 reported cstate change: term changed from 2 to 3, leader changed from dd18efee76df4c2699bff236b9d056ba (127.27.63.66) to b3ae2edef6a0467eaf647236a2c51703 (127.27.63.67). New cstate: current_term: 3 leader_uuid: "b3ae2edef6a0467eaf647236a2c51703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3ae2edef6a0467eaf647236a2c51703" member_type: VOTER last_known_addr { host: "127.27.63.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:07.053093  6198 raft_consensus.cc:1275] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 3 FOLLOWER]: Refusing update from remote peer b3ae2edef6a0467eaf647236a2c51703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:07.054307  6614 consensus_queue.cc:1048] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:07.065063  6277 raft_consensus.cc:1275] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 3 FOLLOWER]: Refusing update from remote peer b3ae2edef6a0467eaf647236a2c51703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:07.066877  6562 consensus_queue.cc:1048] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:07.344352  6277 raft_consensus.cc:1275] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 3 FOLLOWER]: Refusing update from remote peer b3ae2edef6a0467eaf647236a2c51703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:07.345516  6562 consensus_queue.cc:1048] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd18efee76df4c2699bff236b9d056ba" member_type: VOTER last_known_addr { host: "127.27.63.66" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:07.354108  6198 raft_consensus.cc:1275] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 3 FOLLOWER]: Refusing update from remote peer b3ae2edef6a0467eaf647236a2c51703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:07.355759  6562 consensus_queue.cc:1048] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16719ad7c0bd4042acd734beca09a9e0" member_type: VOTER last_known_addr { host: "127.27.63.65" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:07.468199 27901 tablet_server.cc:179] TabletServer@127.27.63.65:0 shutting down...
W20260709 13:15:07.484162  6240 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
W20260709 13:15:07.488826  6240 consensus_peers.cc:597] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba -> Peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Couldn't send request to peer 16719ad7c0bd4042acd734beca09a9e0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:07.493053  6314 consensus_peers.cc:597] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 -> Peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Couldn't send request to peer 16719ad7c0bd4042acd734beca09a9e0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.500501 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:07.501503 27901 tablet_replica.cc:333] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.502424 27901 raft_consensus.cc:2243] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.503098 27901 raft_consensus.cc:2272] T 875130b8f7db4b0faec1c151bc3cf694 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.505549 27901 tablet_replica.cc:333] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.506280 27901 raft_consensus.cc:2243] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.506772 27901 raft_consensus.cc:2272] T 3e6a8dd7983645259cbb2cc1f09c6e7c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.509040 27901 tablet_replica.cc:333] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.509753 27901 raft_consensus.cc:2243] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.510752 27901 raft_consensus.cc:2272] T 4299aff2799e405d8a924eb8227e6fdb P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.512809 27901 tablet_replica.cc:333] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.513372 27901 raft_consensus.cc:2243] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.513764 27901 raft_consensus.cc:2272] T 36330902a8014b269c0ed6ffaf54c238 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.516222 27901 tablet_replica.cc:333] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.517006 27901 raft_consensus.cc:2243] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.517493 27901 raft_consensus.cc:2272] T 4f8b5015e29d44d4a5cbe2d00f415a18 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.519272 27901 tablet_replica.cc:333] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.519769 27901 raft_consensus.cc:2243] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.520181 27901 raft_consensus.cc:2272] T d8171963e8d743b0a09a35917dc3474b P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.522163 27901 tablet_replica.cc:333] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.522645 27901 raft_consensus.cc:2243] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.523013 27901 raft_consensus.cc:2272] T 4456d13ebceb426687e25825f3ba949f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.524902 27901 tablet_replica.cc:333] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.525362 27901 raft_consensus.cc:2243] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.525727 27901 raft_consensus.cc:2272] T 38892ac1ee81491bb340bf1157637db2 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.527585 27901 tablet_replica.cc:333] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.528066 27901 raft_consensus.cc:2243] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.528815 27901 raft_consensus.cc:2272] T 113550147ed34c9ea2d337c2aca91f4e P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.531199 27901 tablet_replica.cc:333] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.532002 27901 raft_consensus.cc:2243] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.532476 27901 raft_consensus.cc:2272] T c41a951e84424e2dbc8f7994aeece051 P 16719ad7c0bd4042acd734beca09a9e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.534850 27901 tablet_replica.cc:333] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.535358 27901 raft_consensus.cc:2243] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.536175 27901 raft_consensus.cc:2272] T 955fe372a4e74944ba40190825e7c20c P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.538165 27901 tablet_replica.cc:333] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.538648 27901 raft_consensus.cc:2243] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.539021 27901 raft_consensus.cc:2272] T 4ba75b94277b4e21adc71d3343f6f0f2 P 16719ad7c0bd4042acd734beca09a9e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.541390 27901 tablet_replica.cc:333] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.541904 27901 raft_consensus.cc:2243] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.542270 27901 raft_consensus.cc:2272] T 89d628bd6ea04eeab2c7dc4a0da5b14f P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.544166 27901 tablet_replica.cc:333] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.544623 27901 raft_consensus.cc:2243] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.545550 27901 raft_consensus.cc:2272] T 6661734bc98940e1b967e62ae244ad39 P 16719ad7c0bd4042acd734beca09a9e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.548334 27901 tablet_replica.cc:333] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.548869 27901 raft_consensus.cc:2243] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.549506 27901 raft_consensus.cc:2272] T 8a10248efa3245f3afbde0526ce1d96d P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.551308 27901 tablet_replica.cc:333] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.551764 27901 raft_consensus.cc:2243] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.552210 27901 raft_consensus.cc:2272] T 824e6d3a7170481b9c540a57c7768843 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.553895 27901 tablet_replica.cc:333] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.554342 27901 raft_consensus.cc:2243] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.555012 27901 raft_consensus.cc:2272] T 99808726cdea4e4897076087e79c92f4 P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.556792 27901 tablet_replica.cc:333] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0: stopping tablet replica
I20260709 13:15:07.557186 27901 raft_consensus.cc:2243] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.558120 27901 raft_consensus.cc:2272] T 3b349dfc09f14259b1cce44dc222096f P 16719ad7c0bd4042acd734beca09a9e0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:07.562260  6240 consensus_peers.cc:597] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba -> Peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Couldn't send request to peer 16719ad7c0bd4042acd734beca09a9e0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.600802 27901 tablet_server.cc:196] TabletServer@127.27.63.65:0 shutdown complete.
W20260709 13:15:07.628707  6240 consensus_peers.cc:597] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba -> Peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Couldn't send request to peer 16719ad7c0bd4042acd734beca09a9e0. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:33369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.637814 27901 tablet_server.cc:179] TabletServer@127.27.63.66:0 shutting down...
W20260709 13:15:07.648479  6240 consensus_peers.cc:597] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba -> Peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Couldn't send request to peer 16719ad7c0bd4042acd734beca09a9e0. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:33369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.662470 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:07.663235 27901 tablet_replica.cc:333] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.663846 27901 raft_consensus.cc:2243] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.664721 27901 raft_consensus.cc:2272] T 36330902a8014b269c0ed6ffaf54c238 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.666639 27901 tablet_replica.cc:333] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.667222 27901 raft_consensus.cc:2243] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.668021 27901 raft_consensus.cc:2272] T 4f8b5015e29d44d4a5cbe2d00f415a18 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.670378 27901 tablet_replica.cc:333] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.671159 27901 raft_consensus.cc:2243] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.672061 27901 raft_consensus.cc:2272] T 875130b8f7db4b0faec1c151bc3cf694 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:07.673379  6316 consensus_peers.cc:597] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 -> Peer dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991): Couldn't send request to peer dd18efee76df4c2699bff236b9d056ba. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.674305 27901 tablet_replica.cc:333] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.675038 27901 raft_consensus.cc:2243] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.675854 27901 raft_consensus.cc:2272] T 3e6a8dd7983645259cbb2cc1f09c6e7c P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.677870 27901 tablet_replica.cc:333] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.678429 27901 raft_consensus.cc:2243] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.678886 27901 raft_consensus.cc:2272] T d8171963e8d743b0a09a35917dc3474b P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.680959 27901 tablet_replica.cc:333] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.681514 27901 raft_consensus.cc:2243] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.682355 27901 raft_consensus.cc:2272] T 4456d13ebceb426687e25825f3ba949f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.684659 27901 tablet_replica.cc:333] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.685563 27901 raft_consensus.cc:2243] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.686365 27901 raft_consensus.cc:2272] T 38892ac1ee81491bb340bf1157637db2 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.688488 27901 tablet_replica.cc:333] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.689023 27901 raft_consensus.cc:2243] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.689459 27901 raft_consensus.cc:2272] T 113550147ed34c9ea2d337c2aca91f4e P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.691388 27901 tablet_replica.cc:333] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.691974 27901 raft_consensus.cc:2243] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.692868 27901 raft_consensus.cc:2272] T 89d628bd6ea04eeab2c7dc4a0da5b14f P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.694967 27901 tablet_replica.cc:333] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.695518 27901 raft_consensus.cc:2243] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.695953 27901 raft_consensus.cc:2272] T c41a951e84424e2dbc8f7994aeece051 P dd18efee76df4c2699bff236b9d056ba [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.698606 27901 tablet_replica.cc:333] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.699144 27901 raft_consensus.cc:2243] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.699612 27901 raft_consensus.cc:2272] T 955fe372a4e74944ba40190825e7c20c P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.701514 27901 tablet_replica.cc:333] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.702052 27901 raft_consensus.cc:2243] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.702466 27901 raft_consensus.cc:2272] T 4ba75b94277b4e21adc71d3343f6f0f2 P dd18efee76df4c2699bff236b9d056ba [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.705109 27901 tablet_replica.cc:333] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.705785 27901 raft_consensus.cc:2243] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.706303 27901 raft_consensus.cc:2272] T 4299aff2799e405d8a924eb8227e6fdb P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.708339 27901 tablet_replica.cc:333] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.708834 27901 raft_consensus.cc:2243] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.709183 27901 raft_consensus.cc:2272] T 8a10248efa3245f3afbde0526ce1d96d P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.710981 27901 tablet_replica.cc:333] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.711416 27901 raft_consensus.cc:2243] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.711756 27901 raft_consensus.cc:2272] T 6661734bc98940e1b967e62ae244ad39 P dd18efee76df4c2699bff236b9d056ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.713548 27901 tablet_replica.cc:333] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.713977 27901 raft_consensus.cc:2243] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.714340 27901 raft_consensus.cc:2272] T 99808726cdea4e4897076087e79c92f4 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.715878 27901 tablet_replica.cc:333] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.716351 27901 raft_consensus.cc:2243] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.716760 27901 raft_consensus.cc:2272] T 824e6d3a7170481b9c540a57c7768843 P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.718462 27901 tablet_replica.cc:333] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba: stopping tablet replica
I20260709 13:15:07.718894 27901 raft_consensus.cc:2243] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.719326 27901 raft_consensus.cc:2272] T 3b349dfc09f14259b1cce44dc222096f P dd18efee76df4c2699bff236b9d056ba [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:07.734268  6316 consensus_peers.cc:597] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 -> Peer dd18efee76df4c2699bff236b9d056ba (127.27.63.66:36991): Couldn't send request to peer dd18efee76df4c2699bff236b9d056ba. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.758747 27901 tablet_server.cc:196] TabletServer@127.27.63.66:0 shutdown complete.
I20260709 13:15:07.789659 27901 tablet_server.cc:179] TabletServer@127.27.63.67:0 shutting down...
I20260709 13:15:07.816185 27901 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:07.816901 27901 tablet_replica.cc:333] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.817803 27901 raft_consensus.cc:2243] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.818436 27901 raft_consensus.cc:2272] T 4299aff2799e405d8a924eb8227e6fdb P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.820384 27901 tablet_replica.cc:333] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.820901 27901 raft_consensus.cc:2243] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.821269 27901 raft_consensus.cc:2272] T 36330902a8014b269c0ed6ffaf54c238 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.822875 27901 tablet_replica.cc:333] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.823367 27901 raft_consensus.cc:2243] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.823714 27901 raft_consensus.cc:2272] T 4f8b5015e29d44d4a5cbe2d00f415a18 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.825539 27901 tablet_replica.cc:333] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.825987 27901 raft_consensus.cc:2243] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.826710 27901 raft_consensus.cc:2272] T d8171963e8d743b0a09a35917dc3474b P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.828689 27901 tablet_replica.cc:333] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.829146 27901 raft_consensus.cc:2243] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.829500 27901 raft_consensus.cc:2272] T 4456d13ebceb426687e25825f3ba949f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.831314 27901 tablet_replica.cc:333] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.831781 27901 raft_consensus.cc:2243] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.832232 27901 raft_consensus.cc:2272] T 8a10248efa3245f3afbde0526ce1d96d P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.833959 27901 tablet_replica.cc:333] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.834517 27901 raft_consensus.cc:2243] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.834985 27901 raft_consensus.cc:2272] T 6661734bc98940e1b967e62ae244ad39 P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.837630 27901 tablet_replica.cc:333] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.838223 27901 raft_consensus.cc:2243] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.838654 27901 raft_consensus.cc:2272] T 875130b8f7db4b0faec1c151bc3cf694 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:07.839841  6314 consensus_peers.cc:597] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 -> Peer 16719ad7c0bd4042acd734beca09a9e0 (127.27.63.65:33369): Couldn't send request to peer 16719ad7c0bd4042acd734beca09a9e0. Status: Network error: Client connection negotiation failed: client connection to 127.27.63.65:33369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.840427 27901 tablet_replica.cc:333] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.841053 27901 raft_consensus.cc:2243] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.841502 27901 raft_consensus.cc:2272] T 3e6a8dd7983645259cbb2cc1f09c6e7c P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.843345 27901 tablet_replica.cc:333] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.843891 27901 raft_consensus.cc:2243] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.844677 27901 raft_consensus.cc:2272] T c41a951e84424e2dbc8f7994aeece051 P b3ae2edef6a0467eaf647236a2c51703 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.846884 27901 tablet_replica.cc:333] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.847452 27901 raft_consensus.cc:2243] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.848279 27901 raft_consensus.cc:2272] T 4ba75b94277b4e21adc71d3343f6f0f2 P b3ae2edef6a0467eaf647236a2c51703 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.851082 27901 tablet_replica.cc:333] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.851620 27901 raft_consensus.cc:2243] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.852257 27901 raft_consensus.cc:2272] T 955fe372a4e74944ba40190825e7c20c P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.854082 27901 tablet_replica.cc:333] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.854566 27901 raft_consensus.cc:2243] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.855003 27901 raft_consensus.cc:2272] T 113550147ed34c9ea2d337c2aca91f4e P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.856808 27901 tablet_replica.cc:333] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.857261 27901 raft_consensus.cc:2243] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.857646 27901 raft_consensus.cc:2272] T 38892ac1ee81491bb340bf1157637db2 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.859493 27901 tablet_replica.cc:333] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.859918 27901 raft_consensus.cc:2243] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.860348 27901 raft_consensus.cc:2272] T 89d628bd6ea04eeab2c7dc4a0da5b14f P b3ae2edef6a0467eaf647236a2c51703 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.862224 27901 tablet_replica.cc:333] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.862636 27901 raft_consensus.cc:2243] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.863340 27901 raft_consensus.cc:2272] T 824e6d3a7170481b9c540a57c7768843 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.865139 27901 tablet_replica.cc:333] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.865576 27901 raft_consensus.cc:2243] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.865934 27901 raft_consensus.cc:2272] T 99808726cdea4e4897076087e79c92f4 P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.867790 27901 tablet_replica.cc:333] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703: stopping tablet replica
I20260709 13:15:07.868367 27901 raft_consensus.cc:2243] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.868818 27901 raft_consensus.cc:2272] T 3b349dfc09f14259b1cce44dc222096f P b3ae2edef6a0467eaf647236a2c51703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.900842 27901 tablet_server.cc:196] TabletServer@127.27.63.67:0 shutdown complete.
I20260709 13:15:07.940429 27901 master.cc:562] Master@127.27.63.126:33459 shutting down...
I20260709 13:15:07.964268 27901 raft_consensus.cc:2243] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:07.964897 27901 raft_consensus.cc:2272] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.965262 27901 tablet_replica.cc:333] T 00000000000000000000000000000000 P db506e36d4b64a448c8a8a7dba0ee3f2: stopping tablet replica
I20260709 13:15:07.986438 27901 master.cc:584] Master@127.27.63.126:33459 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11400 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (23910 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (316261 ms total)
[  PASSED  ] 8 tests.
I20260709 13:15:08.041972 27901 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 409 messages since previous log ~14 seconds ago
I20260709 13:15:08.042174 27901 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 409 messages since previous log ~14 seconds ago
I20260709 13:15:08.042317 27901 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 409 messages since previous log ~14 seconds ago
I20260709 13:15:08.042465 27901 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~32 seconds ago
I20260709 13:15:08.042613 27901 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~32 seconds ago
I20260709 13:15:08.042781 27901 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 6 messages since previous log ~160 seconds ago
I20260709 13:15:08.042968 27901 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 7 messages since previous log ~0 seconds ago
I20260709 13:15:08.043201 27901 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 48 messages since previous log ~10 seconds ago
